Jace Hall's Feed

Jace Hall
03-10-2021 at 09:42 AM
20 Comments
Rate this Entry

The TG Search Engine...

To those that may have interest:

I want to try to improve the TG Search engine.

Based on your experiences with it, what are the main deficiencies that should be addressed?

Any useful information that you can provide would be helpful!

Comments
  1. sdwyer138's Avatar

    I can't rule out that I'm just Doing It Wrong, but when searching for multiple terms, it seems to find each term individually, but I want to be able to get results for only things containing both terms.


    ThanksJace Hall thanked this post
    LikesSnowflake liked this post
  2. Jace Hall's Avatar

    I'm pretty sure @datagod had identified some problems in the past with the TG search system. Hopefully he can weigh in on this...

    Likesdatagod liked this post
  3. datagod's Avatar

    Thanks Jace, I will certainly try to give some suggestions.


    Here we have the results of a search using TG's internal search engine. The results took 16 seconds. Now this is likely not a regular search that TG users would do, but I am just using it as an example.



    A similar search, using a custom google command produced results in under 1 second, and the first link was to this article in my feed. The search term is "raspbery pi" and I added "inurl:twingalaxies" to make sure that only results that have "twingalaxies" in the URL are shown.



    And here is the article I was looking for:

  4. datagod's Avatar

    Several years ago I was experimenting with embedding google search functionality into a website. I created a "SQL Server" specific search page called SQLHunter.com (I retired it this year). This was around 2008. The custom settings for the search bar made sure to give priority to SQL Server results, without the user having to type "SQL SERVER" all the time. There were also quick links to forums, blogs, etc. This was featured in SQL Server Magazine and I made about $1,000 of shared advertising fees.

    Google would insert their ads into the search results, just like they do now. As a host of the page, they shared revenue with me. This paid for the hosting for many years.

    They also had another feature where you let them index your site, and when people use an embedded search bar (like the one at TG now) the results are provided by google with blistering speed.

    A website conducting it's own searches will never be able to compete with a large search engine, unless you are willing to incorporate full text indexing and have a database schema that supports easy searching of the columns (must follow database normalizationg). Alternatively you could go with a document storage database and have rocket fast searching, but that takes a lot of reworking.

    Simple solution? Google embedded search. Share the ad revenue. WIN/WIN

  5. datagod's Avatar

    In less than 60 seconds I made another search engine tailored to TG results.

    https://cse.google.com/cse?cx=7f39056351b389d62

  6. datagod's Avatar

    Very quick results from my new experimental search engine. Notice the "image" and "forum" add-ons. Very simple to do.



    Updated 03-10-2021 at 10:49 AM by datagod
  7. Jace Hall's Avatar

    Quote Originally Posted by datagod

    Very quick results from my new experimental search engine. Notice the "image" and "forum" add-ons. Very simple to do.



    You are correct in that TG will never match google speed. That's not something we are trying to do!

    And while the solution that you suggest does have its merits, using an external system like google does not accomplish all of the things that we hope to accomplish, such as custom topic/area grouping or zero advertisements.

    TG wants to have control over the curation of results so that it can prioritize according to community needs.

    With that being said, your comparative example of google finding and presenting the article you were looking for more suitable than the TG search is quite helpful.

    I think one of the reasons that the "Raspberry Pi" article you were looking for did not immediately show up in TG search is because that article was posted in 2018 and TG search lists most recent first. Its "relevancy" algorithm is limited and could use improvement.

    More of those kinds of functional result differences being highlighted can help inform us on where to improve.

    Also, as people find and identify bugs in our search that is also quite helpful.


    Thanksfrancoisadt thanked this post
    Likesdatagod liked this post
    Updated 03-10-2021 at 12:32 PM by Jace Hall
  8. datagod's Avatar

    I suggest having both. A fast google search AND a customized TG search with grouped results and other options. Speed is key though. We are an impatient species.

  9. BotzaBrand's Avatar

    Maybe include more compilations for titles, if within the same format.

    Steets of Rage on the Genesis is a good example.

    Since it's included on the 6-Pak multicart it could appear in the same search results for Streeys of Rage Genesis / Mega Drive.

    As of now you'd have to search separately for Streets of Rage and/or 6-Pak.

    ThanksJace Hall thanked this post
    Likessdwyer138 liked this post
  10. Jace Hall's Avatar

    Quote Originally Posted by datagod

    I suggest having both. A fast google search AND a customized TG search with grouped results and other options. Speed is key though. We are an impatient species.

    As you have shown, Google is already there for everyone and all know how to access google!

  11. datagod's Avatar

    There is an advantage to having an embedded google search button right on the TG website. Why send people away from the page? Whatever floats your boat I guess.

  12. Jace Hall's Avatar

    Quote Originally Posted by datagod

    There is an advantage to having an embedded google search button right on the TG website. Why send people away from the page? Whatever floats your boat I guess.

    For now we are keeping ads off of TG....

  13. datagod's Avatar

    No ads? That must have been a development when I was off on maturity leave. I am glad to hear it.

    There is a way with the programmable search to monetize ads, but I don't know if NOT doing that means you don't see ads. I doubt it though. Ok, done with google.


    Anyway, using the TG search engine I decided to look up the high score for 1943 arcade. I didn't change anything, just did a default search like anyone would.



    The results also returned a section of unrelated video games. Will upload photo in a bit. I keep getting an error.

    ThanksJace Hall thanked this post
  14. datagod's Avatar

    So by specifying "1943" and "Arcade" I seem to be getting results for both, but not as a single search item.


    Removing "arcade" I can see now there are results over on the right hand side for the games. This is what I was looking for. I wanted to know who had the high score on arcade.

    The grouping is appreciated but it will require a little self-training on my part.


    Updated 03-10-2021 at 04:36 PM by datagod
  15. Rogerpoco's Avatar

    This is NOT a direct answer to your question, but in and around the ballpark, and I have to think it'll be a popular opinion.

    I don't remember exactly how it used to be, but there used to be either "pages", or an alphabetical sorting(select "M", go to the games that start with "M").

    We kinda need that.

    I've looked at MAME some lately, thinking to play it-there are sooo many games.

    It's hard to search sometimes, duplicate games, slight misspelling, I usually wind up just having to "scroll", until I find what I want, I really think an alphabetically sorted page system for systems with a lot of games would help out as much as a search feature, even for those looking for random games to play.

    Likesdatagod, MyOwnWorstEnemy liked this post
  16. thegamer1185's Avatar

    I hate to be that guy, but the search engine on SRC is awesome. Type in "Mario" it pulls up a list relating to all "Mario" games on the site. The more words you type, the narrower the search becomes. It really is a great search system for searching for games only. If TG doesn't want to implement a system like that for whatever reason, I would like to add that adding all games in the search results would be nice. As of right now, no search result directly links to the games for any platform. It shows Accepted scores, submitted scores, articles, posts, but nothing about the actual games involving the search.

  17. RedDawn's Avatar

    I think a relatively easy fix would be to add a "View more" button on the "Games" box on the search page.

    For example, say Im looking for Rampage on NES. If I type in 'Rampage NES' into the search bar, none of the 8 games that are in the 'Games' box on the search page contain any Rampage games.
    If I type 'Rampage' into the search bar, some Rampage games come up, but not the NES one.

    The only other choice this leaves me, that I know of, is clicking TG Records>View Leaderboards>NES/Famicom...and THEN typing 'Rampage' into the secondary search bar.


    When you do a search like in my first or second example, every other box on the search page has a place to click "View more" EXCEPT for the 'games' box, which is limited to only 8 results.


    ThanksJace Hall thanked this post
    Likesdatagod liked this post
    Updated 03-10-2021 at 10:56 PM by RedDawn
  18. RedDawn's Avatar

    Unfortunately the TG search bar simply doesn't work for 50% of my searches. Probably more. It can be very frustrating.
    When its happened lately I harken back to this post, so that is why I'm commenting now.

    I also like how the search page is grouped into different categories, but there has got to be a better system than there is presently. I dont know how this would be implemented, but maybe if there was some way to specify whether your searching for a game, person, wall feed, forum topic, wiki etc

    I think some of the problems may be: (1) the limited number of results, (2) what results are given priority over other results, (3) the spelling requirements, and (4) how the search results are full of longer terms which contain the search term as part of a larger word.

    For example, I was watching the KoK extra features and saw a segment where RTM was naming the top 5 gamers of the period. I didnt recognize one of the names, 'Anders Svenson'
    so I searched it at TG and all the results were like 'steve sANDERS' 'tom ANDERSon' 'defeat the commANDERS' etc but NOTHING related to this gamer I was searching for.

    I went through about 8 different spellings of his last name before hitting on 'Svensson', the correct spelling of his name, and when I did 90% of the results were a recent post by JJT asking what happened to several gamers of TG's past. I did find his profile on the page, but this was all after a great deal of effort, relatively speaking. Looking up all the games and high scores for the Jonas Neubauer Memorial BTC Bounty was no small endeavor. It took me hours when it could have taken mere minutes had the search engine been more accommodating.

    I dont know how programing a search engine works, the TG search engine seems like it needs to be more specific in some areas while more forgiving in others, but these are some of the issues I have found in my experience with it. Hope it helps.

    Thanksdatagod thanked this post
    Updated 03-29-2021 at 07:59 AM by RedDawn (clarification)
  19. datagod's Avatar

    Quote Originally Posted by RedDawn
    I went through about 8 different spellings of his last name before hitting on 'Svensson', the correct spelling of his name, and when I did 90% of the results were a recent post by JJT asking what happened to several gamers of TG's past. I did find his profile on the page, but this was all after a great deal of effort, relatively speaking. Looking up all the games and high scores for the Jonas Neubauer Memorial BTC Bounty was no small endeavor. It took me hours when it could have taken mere minutes had the search engine been more accommodating.



    Using Microsoft Bing yields the correct spelling. Search Engines are highly sophisticated and have many features not easily replicated. I am sure Google does the same.

    No, I am not beating a dead horse. Well, maybe.

    ThanksJace Hall thanked this post
  20. Blackflag82's Avatar

    This conversation comes up in the library world from time to time...Librarians hear from patrons that the search tool doesn't work well. We librarians get on a variety of list serves and talk about ways the library discovery tools could be better, how it could be tailored to specific libraries or library types, how development could take place, etc... But at the end of the day, it boils down to one simple fact - The libraries that use Google (or some other search function) embedded on their library page (and in the best of cases, specifically tailored to showing their results first), don't get these complaints from patrons with nearly as much frequency, and they didn't spend vast resources on a subpar search function....

    Likesdatagod liked this post
Join us