Frege is concerned with how to explain how the parts of a proposition "hold together". To do this, Frege thinks that we need a distinction between the sense and the referent of a predicate on the one hand, and those of a proper name (singular terms and sentences) on the other.
1. There predicates.
2. There are proper names.
3. The there are senses.
4. There are referents.
We can conclude immediately, given how the terms are introduced:
5. Predicates and proper names are distinct.
6. Senses and referents are distinct.
This means that we can conclude:
7. A predicate has a sense and a referent.
8. A proper name has a sense and a referent.
Frege also holds that:
9. The sense of a predicate is unsaturated.
10. The referent of a predicate is a concept.
11. The sense of a proper name is saturated.
12. The referent of a proper name is an object.
But despite this, it remains a "vexed question" whether or not saturated senses can ever be referents of proper names and so qualify as objects. We certainly have reason to think that unsaturated senses cannot be referents of proper names and so cannot be objects.
This means we have reason to believe the truth of (13):
13. Proper names cannot have as referents unsaturated senses, so unsaturated senses cannot be objects.
But we have to take seriously that:
14. If proper names can have as referents saturated senses, then saturated senses qualify as objects.
But it is not clear that Frege would allow that we can have a proper name which refers to the sense of a proper name, or that we can have saturated senses which are objects. The motivation for (5) would seem to be the same as (6): that something that is a sense is logically ruled out as being a referent, just like something that is a predicate is logically ruled out as being a proper name. But it seems that we could have a sense which is made a referent, even though we cannot have a referent which is made a sense.
Given this rough view, Frege starts by taking singular terms and sentences as basic features of language, characterizing predicates as being an expression obtained by removing one or more occurrences of singular terms.
15. Singular terms and sentences are basic features of language.
16. We can derive predicates as expressions by removing one or more occurrences of singular terms.
17. If we remove one or more occurrences of singular terms, we are left with "gaps" in our language.
Thus, we have something with gaps which need to be filled out to form a sentence.
But we must be careful not to think that the sense of a predicate is something which stands in need of similar completion. Frege retreats to the idea of an unavoidable "awkwardness of language" that limits us severely when we try to apply language to its limits, since we only have meager linguistic resources which we are trying to apply to language itself.
This would seem to set up a reductio of Frege's project.
(18) Frege deploys the resources of language to explore language.
(19) The resources of language are inadequate to fully explore language.
(20) So Frege's theory is unable to adequately and fully explore language.
But Frege sees this "fog of paradox" as nothing against the doctrine engulfed in it, but due to the unspeakable depths being plunged. We just have to accept that language has limitations which inhibit an exploration of its own foundation.
It seems to me that Frege wants to have a nice distinction and clear cut between predicates and proper names, hence (5); and that he would like this nice distinction and clear cut to be mirrored between senses and referents, hence (6). But this nice distinction and clear cut seems to be lacking, since I might refer with a proper name to a sense, thus having something that is a sense, but that is also a referent to something else. And Frege just seems to think this is a general problem when we use language against itself.
This reminds me of two general notions: those of Rorty and those of Wittgenstein's Tractatus.
Rorty attacks metaphysical doctrines, particularly foundationalism and logical atomism, as introducing a notion of "B class entities" which are only meaningful insofar as they are reducible to "A class entities". So everything that is meaningful is reductive to a common base. But, oops, "A class entities" are not reducible to themselves. So the Tractatus makes this sort of mistake by holding that meaningful sentences reduce to atomic sentences, which themselves are not reducible. And hence not meaningful? No, they have to be meaningful in a special way. For Rorty, this shows how flawed and ridiculous the whole enterprise was in the first place. But Frege sounds here like the Wittgenstein of the Tractatus. It's just a general problem with the nature of our investigation, not with the specifics of our investigation itself.
The Wittgenstein of the Tractatus was very Fregean, and I see some interesting parallels. Perhaps in Frege's defense we could appeal to the Tractatus' quietism. There is only so much work that saying can do. The foundation of things is a showing, which is more rich than a saying. Instead of trying to talk about the limits and foundations of language, we should rather realize that we can never have any such meaningful talk; and then this shows us something. We can try to talk and fail, hence the "fog of paradox" due to the unspeakable depths being plunged. But then this "fog" and the "unspeakable depths" make themselves manifest by showing themselves, not by being explicitly said.
So the natural response to Rorty is that "B class entities" are the 'sayers' and the "A class entities" are the 'showers'. So of course the sayers get reduces to the showers, and the showers are irreducible. But then again, this may just seem like a reductio of this way of looking at things. The Wittgenstein of the Tractatus thought it was, and that a Fregean way of looking at things is only useful for bringing up the problems it does in order to show us something unsayable. This was not something Frege understood at all, and if he did understand it I am sure he would not be at all happy with that interpretation.