Usually phone wires are red, green, yellow and black. Currently you typically need only the red and green wires to make a phone work. There are several things that can be wrong.
1. Who ever installed the phone did not follow standard color conventions on the wiring.
2. The wiring is disconnected at the other end.
3. The wiring is just defective.
4. It's not phone wiring at all.
Was the previous owner a computer user? These wires could be for DSL or ethernet computer connections. Can you find the other end and run a continuity test on the wires and verify they are actually connected?