Project 1
Questions and Answers

CS 422/5422 - A. Hornof - January 26, 2024

These are questions from students, and my responses. New additions will be added at the top.

February 1, 2024

Q: Can we use the python "requests" module in our projects? It is described here: https://pypi.org/project/requests/

A: Yes

January 26, 2024

Q: The project handout says that "discrepancies between names found in 'gradedata.js' and the names found in the scraped instructor data would be easy to resolve using the administrator tools, to ensure that the data in your tables is clean, consistent and accurate." (I added the underlining.)

Does "easy to resolve" mean that there should be a graphical user interface to assist the administrator in resolving the discrepancies?

A: No, it just means that there should be a straightforward process to resolve the discrepancies, a process that does not require the administrator to study item-by-item through any list. For example, "the tools could generate statistics such as lists of names from both data sources that have yet to find a match, so you can see how your data resolving process needs to be further improved." This would assist the administrator in easily finding and editing the names in one of the input lists to remove discrepancies, and to then rerun the code that combines the faculty and grade lists, and to repeat this process until there are no more discrepancies. (The process for removing discrepancies should be documented.)

Q: Can we edit the provided gradedata.js file before running? Are we able to manually edit and/or write code that will alter it, such as removing the functions at the bottom?

A: I would expect that the administrator part of the system that reads in the "gradedata.js" file, would read the file in its current form, without any manually pre-editing of the file. Yes, it would make sense for the code that reads in the .js file to ignore functions that appear in the .js file. Yes, you can write code that will alter the file, though it would seem wise to not overwrite the original input file. (One imagined "use case" is that you could, in the future, be given new gradedata.js files in a similar format, and it would be good for your system to work with those files with minimal modification to the current system.)

Q: When scraping the faculty names from the Wayback machine, what do we do with the different types of faculty that appear under different headings in some of the departments' lists of faculty? Such as "Faculty", "Emeriti", and "Participating"? For example, the "African Studies" section has two sections in its faculty tab: "Participating faculty" and "Associated".

A: Please include all faculty under all headings as part of the "Regular Faculty" (as the term "Regular Faculty" is used in the project handout). (The reason is this: Some classes are offered under "Subject" codes that do not have a department associated with them. These classes are taught by faculty with other "home" departments. For example, all of the "AFR - African Studies" classes are taught by faculty with other "home" departments.)

January 25, 2024

Question 1: The Project 1 handout states that "The data file (that the system administrator will use to load the new grade data) can be in a file format of your choice, such as the original “gradedata.js” file that is provided to you for this project, or a CSV file." Does our system have to use the grade data from the “gradedata.js” file that is provided to you for this project?

Answer: Yes. You may find a .cvs file with some overlapping grade data on the github site associate with the Daily Emerald system. You may use this file however you like, but you must ensure that the resulting data perfectly match the data in the “gradedata.js” file that is provided. So, yes, you must in some way use the “gradedata.js” file that is provided to you for this project.

The reason for this this, by the way, is that this provides a means to validate the output of your systems, in that the outputs can be compared to the output of the Daily Emerald system.

January 15, 2024

Question 1: The Project 1 handout states that "No server connections may be required for either installing or running the software, except for setting up mysql on ix, and you must provide instructions on how to set this up."

So, are we allowed create a web application, such as using Javascript?

Answer: Your system can use a server and Javascript provided that your submission provides all of the files, and instructions, necessary for someone (with moderate Unix expertise) to set up the system from scratch on ix.cs.uoregon.edu. In other words, please do not submit a URL to a server that your group has set up, and direct the person installing the software (me) to that URL, either to see the code running, or to download anything. Similarly, your code should not access any servers or databases on the internet, unless those servers or databases are on ix, and you provide instructions on how to set up those servers or databases.

Question 2: Can we break up the administrator's tasks into smaller steps (such as scraping, processing, and installing), and the software permits the administrator to only do the steps necessary at the time they are using the software? For example, once the administrator scrapes and processes, perhaps they never return to those steps.

Answer: That is a good point that there are multiple steps to the administrator's tasks, which (at least) include scraping the data, processing the data, and installing the system. It would make sense for your system to permit an administrator to do some of these things independently of the others. For example, after the administrator scrapes the data, provided the data do not change, there should be no need for that administrator to scrape it again. But please do include the complete administrator functionality, including scraping and processing, with your submission.

Question 3: Will we continue to update our SDS throughout the course of the project, or is the initial SDS the final product?

Answer: Please do update your SDS as you go, so the SDS correctly characterizes the intermediary and final system designs. This class (CS 422/522) supports the idea that designs should evolve, and that your software development process model should include the steps necessary to record the history of the design process, as well as a clear statement of the current design.

Question 4: Can we use BeautifulSoup for our web scraping?

Answer: Yes!