Saffron, that most exotic and expensive of spices can be grown commercially only where the cost of the intense labor needed for harvesting can be provided cheaply enough to make the crop cost effective. Much of the modern supply comes from Iran, India, and Spain, but it has a long history having been cultivated in Crete and ancient Egypt. An English town, Saffron Waldon, was so named because it was the center of cultivation in that country during the 16th century.
However, given the right conditions it can be easily and successfully grown in USA zones 5-8 which covers much of the Owens Valley, but one has to be prepared for the challenge of harvesting which is a tedious business! A local gardener grows them successfully in Bishop Community Garden, and even if you don't use them for saffron the flowers are pretty.
The orange threads of the spice are the stigmas (not the stamens) of Crocus sativus, a Fall blooming crocus with violet flowers and yellow stamens. Three long orange stigmas are held in each flower. The flowers are picked as soon as they are well out and are carefully cut open, which enables the stigmas to be extracted with tweezers. It is recommended that these be placed in a dish or on a paper towel to dry after which they should be kept in an airtight container in a dark place since sunlight degrades them. Apparently they do need to be dried before use. When I grew them I tried using them fresh, and found the flavor lacking.
Many catalogs now offer these corms, which should be planted during the dormant season. It takes 75,000 stigmas to produce one pound of saffron, so it is difficult to recommend how many the home gardener should begin with since it depends on expected usage and space available, but it is generally suggested that one should start with about two dozen corms. Make sure that you buy only Crocus sativus which should not be confused with other fall blooming crocus species, or the autumn-flowering Colchicum which is toxic. The corms are fairly large for a crocus, and are dormant in the summer and then send up flowers and narrow grassy leaves as the weather cools — around the end of September in Bishop. After blooming the leaves continue to grow for a time — they are frost hardy — and the bulbs multiply before the plants die down when temperatures heat up the following year. At this point they should be kept fairly dry because they tend to rot if wet, but occasional irrigation might be warranted in our desert climate.
They prefer sandy soil, and should be planted when dormant, about 4” deep and 4” apart in a spot which receives about 6 
Interestingly Crocus sativus is an entirely cultivated species which does not occur in the wild. The corms multiply rapidly under ideal conditions, but it is a triploid so it does not set seed since its chromosomes can't pair up properly.
Not only are the stigmas used as a flavoring, but they are also used as a dye for fabric, for making perfumes, and in some places they are used in herbal medicine.
Why not give it a try?