resetreboot
/
tfidf
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
A simple tfidf crawler and search engine. Mostly as a proof of concept.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
2 Commits
1 Branch
0 Tags
156 KiB
 
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
Go to file
José Carlos Cuevas 3363716124
Pushing a new README.md 		9 years ago
src/tfidf Initial commit 9 years ago
.gitignore Initial commit 9 years ago
README.md Pushing a new README.md 9 years ago
main.go Initial commit 9 years ago
test.go Initial commit 9 years ago

README.md

TfIdf Crawler in Go

A simple website crawler and tfidf parsing engine to implement a simple search engine.

Command Line Parameters

This utility relies in a couple command line arguments.

  • -w: Allows to choose the file where the webs to index will be found. If not present, it assumes a websites.txt file in the same directory the command is run in.
  • -s: Chooses a stopword language. It assumes english1 as default. See the stopword files for choices.

Known bugs

  • Stopwords should be loaded from the lib directory or implement a path search.
  • Probably I messed up some of the tfidf calculations.