समस्या
Language models work with tokens, not raw text. A simple tokenizer lowercases the text and treats every run of letters a-z and digits 0-9 as a word; everything else (spaces, punctuation, other characters) separates words.
Build the vocabulary: the distinct words, sorted by how often they appear (most frequent first), with ties broken alphabetically. A word's position in this list is its token ID.
Implement build_vocabulary(text).
Input
text: Any text.
Output
Return the vocabulary as a list of words. Return an empty list if the text has no words.
सीमाएँ
0 ≤ length of text ≤ 100,000
उदाहरण
इनपुट:
text = "The cat sat on the mat. The end!"
अपेक्षित आउटपुट:
expected = ["the", "cat", "end", "mat", "on", "sat"]
इनपुट:
std::string text = "The cat sat on the mat. The end!";
अपेक्षित आउटपुट:
std::vector<std::string> expected = {"the", "cat", "end", "mat", "on", "sat"};
इनपुट:
string text = "The cat sat on the mat. The end!";
अपेक्षित आउटपुट:
string[] expected = new string[] {"the", "cat", "end", "mat", "on", "sat"};
व्याख्या
Lowercased, the words are the, cat, sat, on, the, mat, the, end. "the" appears 3 times; the rest appear once and are sorted alphabetically: ["the", "cat", "end", "mat", "on", "sat"].
इनपुट:
text = "AI, ai and A.I. (2024)"
अपेक्षित आउटपुट:
expected = ["ai", "2024", "a", "and", "i"]
इनपुट:
std::string text = "AI, ai and A.I. (2024)";
अपेक्षित आउटपुट:
std::vector<std::string> expected = {"ai", "2024", "a", "and", "i"};
इनपुट:
string text = "AI, ai and A.I. (2024)";
अपेक्षित आउटपुट:
string[] expected = new string[] {"ai", "2024", "a", "and", "i"};
व्याख्या
Words: ai, ai, and, a, i, 2024. "ai" appears twice; the others once, sorted alphabetically with digits before letters: ["ai", "2024", "a", "and", "i"].
कैसे हल करें
पहले खुद कोशिश करें, फिर एक-एक करके संकेत खोलें।
संकेत 1
First extract only runs of letters and digits, lowercased, and ignore every other character.
संकेत 2
Count how often each word appears, then sort by frequency descending and alphabetically for ties.
संकेत 3
If the text contains no valid words, return an empty list instead of a singleton or null result.
अपेक्षित जटिलता
O(n log k) time, O(k) extra space
संदर्भ
यह चुनौती CodeUtility के लिए अनुकूलित की गई है। ऊपर दिए गए स्टेटमेंट और सीमाओं का पालन करें।