GitHub - demid5111/nlp: make tool for expert advising

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 27 Commits
frontend		frontend
.gitignore		.gitignore
NLPresults1.csv		NLPresults1.csv
NLPresults2.csv		NLPresults2.csv
README.txt		README.txt
abase.sql		abase.sql
authors.list		authors.list
build_lda.py		build_lda.py
categories.list		categories.list
collect_categories.py		collect_categories.py
collect_info_for_lda.py		collect_info_for_lda.py
dict.pickle		dict.pickle
hse_model.lda		hse_model.lda
hse_model.lda.state		hse_model.lda.state
log.txt		log.txt
logfile2.txt		logfile2.txt
nlp_db_dump.sql		nlp_db_dump.sql
nlp_models.py		nlp_models.py
post.py		post.py
process.py		process.py
publications.txt		publications.txt
readCSV.py		readCSV.py
sample.dict		sample.dict
sample.mm		sample.mm
sample.mm.index		sample.mm.index
sample.py		sample.py
scan.py		scan.py
test_count.py		test_count.py
test_sample.py		test_sample.py
update.py		update.py

Repository files navigation

abase.sql - MySQL dump for fast creation of DB for the articles info

scan.py - script for visiting web-pages, parsing them and getting names of authors

collect_info_for_lda.py - reads the file logdile.txt and gets authors names and their publications

sample.py - main script, when being called as script calls computing function for a "text"

logfile.txt - results of scannong all known from DB webpages

authors.list - serialized author and all text tokens list

hse_model.lda - serialized tfid lda model for eliminationg building it online

hse_model.lda.state			|DO NOT REMOVE
sample.mm.index 			|service files

sample.mm - serialized mmCorpus

sample.dict - serialized corpora.Dictionaty for lda model