Determination of selective and nonselective compounds for the 5-HT(1A) and 5-HT(1B) receptor subtypes in rat frontal cortex

Recent studies indicate that there are multiple subtypes of the 5-hydroxytryptamine₁ (5-HT₁) receptor. Previously, we provided evidence consistent with the finding that multiple states of the 5-HT₁ receptor are present when the binding of [³H]-5-HT is measured in the absence of guanine nucleotides. When 1 mM GTP was present in the [³H]5-HT receptor binding assay, the high affinity state was eliminated. As the presence of multiple states of a receptor complicates the interpretation of the inhibition of [³]-5-HT binding caused by serotonin agonists and antagonists, we examined the ability of a series of these drugs to compete for 15 nM [³H]-5-HT binding in the presence of 1 mM GTP in the rat frontal cortex. Eight agonists and five antagonists showed selectivity for the two subtypes of the 5-HT₁ receptor, whereas three agonists and four antagonists showed the same affinity for these two receptors subtypes. Most of the compounds examined exhibited only a modest 10- to 30 fold degree of selectivity. However, 1-(m-trifluoromethylphenyl) piperazine and 5-methoxy-3-(1,2,3,6-tetrahydro-4-pyridinyl)indole were about 65-fold selective and spiperone was over 100-fold selective for one of the receptor subtypes. The subtype specificity of the selective compounds was determined using either spiperone, a selective 5-HT₁A) compound, or 1-(m-trifluoromethylphenyl)piperazine, a selective 5-HT(1B) compound, to preferentially inhibit one of the receptors. The results of these experiments indicate that spiperone, pizotifen and the indole agonists 5-methoxy-N,N-dimethyltryptamine and N,N-dimethyltryptamine are selective for the 5-HT(1A) receptor, whereas 5-methoxy-3-(1,2,3,6-tetrahydro-4-pyridinyl)indole and the piperazine agonists 1-(m-trifluoromethylphenyl)piperazine, 1-(m-chlorophenyl)piperazine and quipazine are selective for the 5-HT(1B) receptor. No antagonists selective for the 5-HT(1B) subtype were identified. These results indicate that selective indole agonists show a higher affinity for the 5-HT(1A) receptor and piperazine agonists demonstrate selectivity for the 5-HT(1B) receptor. Also, the data are consistent with there being only two subtypes of the 5-HT₁ receptor in rat frontal cortex.