joshnie/KU-courses
1
0
Fork 0
Code Issues 4 Pull requests Projects Releases Packages Wiki Activity Actions
A better version of KU's course catalogue https://kucourses.dk
861 commits 6 branches 0 tags 14 MiB
Find a file
Exact
Joshua Niemelä 136fbf1ce9
Some checks failed
Deploy to server / Deploy to server (push) Failing after 7s
Details
Rust CI / Rust quality check (push) Failing after 9s
Details
Rust CI / Rust quality check-1 (push) Failing after 7s
Details
temp
2025-11-27 16:52:29 +01:00
.github/workflows Format and fix container labels 2025-11-16 21:37:56 +01:00
assets added gif 2023-07-04 10:11:34 +02:00
backend temp 2025-11-27 16:52:29 +01:00
frontend Standardise docker for frontend 2025-11-23 11:32:31 +01:00
.envrc bla 2023-05-11 19:50:19 +02:00
.gitattributes Update .gitattributes 2023-10-16 22:18:40 +02:00
.gitignore added git ignore 2023-08-25 11:56:27 +02:00
.pre-commit-config.yaml Added another precommit 2023-06-20 16:49:12 +02:00
CONTRIBUTING.md Added contributing md 2023-06-15 00:53:25 +02:00
docker-compose.yml Experiment with docker compose settings for dns bug 2025-11-23 11:36:28 +01:00
flake.lock fixed flake 2023-12-24 16:13:30 +01:00
flake.nix Update vector store 2025-11-23 13:57:41 +01:00
LICENSE Credits 2024-01-10 15:46:03 +01:00
README.md Update README.md 2025-05-18 12:25:45 +02:00

README.md

KU-Courses

Example of KU-Courses

The entire application is governed through the docker-compose.yml file and is built with docker compose:

Starting the application

  1. Install docker and docker-compose, this may need a restart of your system since Docker is a very low level program.
  2. Run docker compose up --build as either a user with permissions to docker, or with sudo/doas, the build flag is required if the backend or frontend code has been changed, additionally -d will make it detach from the terminal. a. Add folders exam_pdfs and pages to the data folder.
  3. Wait for the scraper in the backend to complete scraping pages, this may take about 15 minutes.
  4. Run docker compose restart, this is required so that the parser will run and so that the vector store can create new embeddings.
  5. ???
  6. PROFIT!!!

db-manager

The backend is built with Clojure, a functional programmering language based on Lisp which runs on the Java Virtual Machine.
This part serves multiple purposes, it is responsible for scraping the course pages from KU as well as the statistics from STADS.
The backend also serves the frontend and contains the "datascript" database and is responsible for refreshing and various services occasionally (this feature is partially broken at the moment).

vector_store

This service is responsible for the semantic searches used in the get_course_overviews route, instead of using trigrams or full-text, we decided to use vector searches for the lower latency.

rust_parser

This service is the parser that takes the scraped course pages and parses them into a format we can use in the database for searching and for serving to the frontend.

frontend

Frontend is built in Svelte/Typescript. This is a highly responsible SPA that shows the courses in the form of cards which can be clicked into to get a more detailed view of the course.

Credits

  • Thanks to Jákup Lützen for creating the original course parser in Python.
  • Thanks to Kristian Pedersen for creating the original frontend, and help in designing the architecture and first database schema.
  • Thanks to Zander Bournonville for creating the statistics parser.