How can I downgrade my version of PHP ?

Iphp 5.3 on Fedora 17 or Redhat using atomic repo.

You will need to remove php and httpd first (not compulsory)

yum remove php
yum remove httpd

Add the following at the end of /etc/yum.conf(if required)

exclude=php*5.4*

Install php from atomic repo

wget -q -O - http://www.atomicorp.com/installers/atomic | sh
yum install php

More details at 
http://www6.atomicorp.com/channels/atomic/fedora/17/x86_64/RPMS/