Ref/search!

For the search, it's actually two find operations (probably because I haven't looked into how MongoDB handles join operations yet). First you get the entries from `articles_search` so you can have the `articles._ids`; second you get the article content. Some of my code to get you through the relatively hard parts: ### First, a Useful Utility Method ```js _renderSearchTerms(theString) { return theString .replace(/\(/gi, ' ') .replace(/\)/gi, ' ') .replace(/\./gi, ' ') .replace(/!/gi, ' ') .replace(/@/gi, ' ') .replace(/#/gi, ' ') .replace(/\$/gi, ' ') .replace(/\%/gi, ' ') .replace(/\^/gi, ' ') .replace(/&/gi, ' ') .replace(/\*/gi, ' ') .replace(/-/gi, ' ') .replace(/_/gi, ' ') .replace(/=/gi, ' ') .replace(/\+/gi, ' ') .replace(/\{/gi, ' ') .replace(/\[/gi, ' ') .replace(/\}/gi, ' ') .replace(/\]/gi, ' ') .replace(/:/gi, ' ') .replace(/;/gi, ' ') .replace(/"/gi, ' ') .replace(/'/gi, ' ') .replace(/`/gi, ' ') .replace(/,/gi, ' ') .replace(/>/gi, ' ') .replace(/</gi, ' ') .replace(/\//gi, ' ') .replace(/\?/gi, ' ') .replace(/\|/gi, ' ') .replace(/\\/gi, ' ') .replace(/\n/gi, ' ') .split(' '); } ``` The above utility method: - removes all the unwanted characters from a string you intend to index for search, replacing them with spaces - splits the result into an array, using a space as the split delimeter - returns it ### Whenever you Add or Update an Article, Set the Keywords Info in `articles_search` Next is the method that sets the `articles_search` entries for an article. ```js async setSearchTerms(arg) { const d = mdb.db("amgdotcom"); const t = d.collection("articles_search"); let id = (arg._id !== undefined) ? arg._id : arg.id; if (id === undefined) return false; await this.deleteSearchTerms(id); ``` We don't want duplicate entries, so we've deleted all the old search terms for this article. ```js let terms = [ ...this._renderSearchTerms(arg.title), ...this._renderSearchTerms(arg.content) ]; ``` This creates a merged array; note that there may very well be words in the `title` that are also in the `content` ...that's fine, because right now the `terms` array contains the entire article as an array broken into words -- no sums yet, just a raw list of words exactly as they appeared in the original article. Not until we... ```js let termCount = {}; terms.forEach((item) => { item = item.trim().toLowerCase(); if (item !== '') { if (termCount[item] === undefined) termCount[item] = 1; else termCount[item]++; } }); ``` *Now* we have the sums. Next we simply collate a `docs` array full of objects compatible with MongoDB's node driver method, `[db].[collection].insertMany()`. ```js let docs = []; // rewriting terms var terms = Object.keys(termCount); terms.forEach((item) => { docs.push({ article_id: ObjectId(id), term: item, count: termCount[item] }); }); ``` Finally we insert the `articles_search` entries into the collection (`t` was defined all the way on the 2nd line of this function). ```js t.insertMany(docs); } ``` ### And Finally, the Search Method ```js async searchArticles(term, type='blog') { ``` The `term` parameter needs to be a space-separated string of keywords. ```js try { let r = []; let article_weights = {}; const d = mdb.db("amgdotcom"); let t = d.collection("articles_search"); ``` Just setting up variables to be used later. The collection `t` will be reassigned later, so it's not a `const` like it otherwise would be. ```js let query = { $or: [] }; let terms = term.split(' '); for (let i = 0; i < terms.length; i++) { query['$or'].push({term: new RegExp(terms[i], 'gi')}); } ``` We're dynamically building a `query` object, above. ```js let options = { sort: { count: -1 }, projection: { article_id: 1, count: 1 }, }; ``` All we need from the `articles_search` collection is the `articles._id` value stored in `articles_search.article_id`, and the `count` of how relevant the keyword was to that article. We don't care about the keyword, but we do sort by `count`, descending. ```js const cursor1 = await t.find(query, options); await cursor1.forEach((item) => { if (!article_weights.hasOwnProperty(item.article_id)) article_weights[item.article_id] = { count: item.count }; else article_weights[item.article_id].count += item.count; }); ``` Above, we're priming the `article_weights` object now, calculating the total relevance. The goal is to know how relevant the article is when you take all keywords in the search term into account at once. ```js // resetting query if (type === 'all') query = { $or: [] }; else query = { type: type, $or: [] }; let article_ids = Object.keys(article_weights); for (let i = 0; i < article_ids.length; i++) { query['$or'].push({_id: ObjectId(article_ids[i])}); } ``` Once again, we're building a dynamic query above. This time we want every article that's relevant at all. ```js // resetting t t = d.collection('articles'); // resetting options options = { projection: { _id: 1, title: 1, content: 1, tags: 1, username: 1, creation: 1, updated: 1, type: 1, }, }; ``` Now we're preparing to do a `.find()` on the `articles` collection. I didn't bother sorting on the lazy assumption that I have to do that manually. (Seriously, I should just look up how MongoDB joins work, if at all.) ```js const cursor2 = t.find(query, options); ``` I love the MongoDB Node.js driver so far. ```js await cursor2.forEach((item) => { r.push({ id: item._id, title: item.title, content: item.content, tags: item.tags, username: item.username, creation: item.creation, updated: item.updated, type: item.type, }); }); ``` The `r` array is the return value. Here we're packing it full of everything needed to render each article that was relevant to the search. ```js r.sort((a, b) => { if (article_weights[a.id].count > article_weights[b.id].count) return -1; else if (article_weights[a.id].count === article_weights[b.id].count) return 0; else return 1; }); return r; ``` Above, we sort the return value so that the more total keyword matches for an article, the earlier in the array it'll be sorted. ```js } catch (e) { console.log(`There was an error: ${e}`); } } ``` The other methods are easy, basic CRUD stuff. Have fun with your fancy new relevance-weighted search!