Project 1
This project aims to introduce students to data scraping and indexing in SOLR. Before mining and
analyzing
such content, collecting, cleaning, and storing such data is a crucial step. This project aims to
familiarize students with collecting online data and efficiently storing them, making data retrieval and
analytics easier for downstream applications. This project will also introduce students to the different
technical aspects involved in this course and subsequent projects.
Download: Project 1
Project 2
In this project, you will be given a sample input text file consisting of Doc IDs and sentences. Based on this provided input text file, your task is to build your own inverted index and host it as a Flask app for querying. Your index should be stored as a Linked List in memory as the examples shown in the textbook (refer to Chapter 1 – Boolean Retrieval). Having built this index, you are required to implement a Document-at-a-time (DAAT) strategy to return Boolean query results. Your implementation should be based only on Python3.
Download: Project 2
Project 3
The final project aims to integrate the skills and concepts learned in the first two projects into a comprehensive Information Retrieval (IR) chatbot.
Project 1: Indexing and Crawling - Focused on gathering data from Wiki Pages, students learned to index this data effectively using Apache Solr.
Project 2: Scoring - Explored the mechanics of building your own inverted index from scratch and retrieving documents using AND queries.
Project 3: End-to-End IR Chatbot.
There are two different topics for the final project. Select any one of them (Topic 2 contains additional bonus points if attempted)
Difficulty Level: Topic 1 is easier compared to Topic 2
Download: Project 3