EDIT:--Good info OM....I did not know that...Sounds like fun...I am staring this question and putting that on my TTD list...
Although the LA Equestrian Stables have excellent rental horses; my favorite stable to rent from are the Circle K Stables on Mariposa; just west of the Equestrian Center and two blocks south of Riverside Drive. The stables are open every day of the year except Christmas day; from 7:30 in the morning until dusk. On special occasions they have evening rides. Fees are 15 dollars for the first hour and 10 dollars for each additional hour. There is a 25 dollar deposit.
On the Hollywood side of Griffith Park sits the Sunset Ranch Hollywood Stables. Located up picturesque Beachwood Canyon literally under the Hollywood sign, one can take evening equestrian caravans through Griffith Park to the Viva Fresh Mexican restaurant for dinner; the return ride gets you back to the stables at midnight. This is a different sort of an evening; you just might get to hear a coyote howling. Sunset also offers riding lessons, lunch rides and horse rentals at competitive rates.