I'm guessing that you have vinyl, not linoleum. The difficulty is that dust gets under the vinyl and makes it hard to glue back down. But there are products especially for this repair, and those I have heard of actually glue the edges of the vinyl together.
I think you need to go to a flooring store, but a home center may have this product.