CIS 422/522 Software Methodologies
Project Evaluation Criteria

A. Hornof - 4-24-2018

Final Project Delivery

Final projects should be delivered by means of one team member depositing a single .zip file of all project documents and files into Canvas before date and time that the project is due.

Unfortunately, some projects receive lower grades than they should simply because it is not clear where to find some of the team's work products, work products are missing, or it is not clear exactly how to run and use the application. Please make sure that you have multiple team members review the final .zip that will besubmitted well before the final date and again before delivery. The following checklist is not complete but may help avoid some of the more obvious errors:

1. Completeness: Make sure that every piece of the project is completed. Make sure that there is a final sign-off on each work product by both the writer and a second person acting as reviewer.
2. Consistency: Where documents depend on one another, have the documents reviewed for consistency. For example, make sure that the User's Guide is consistent with the actual code delivered and with the requirements.
3. Application: It is critical that you provide all of the instructions and links necessary for anyone to set up (if needed) and run your application. I suggest testing this on users who are completely naive to your project. The teaching staff cannot award points for code that they cannot run and will not have time to try to debug code or installation instructions.
4. Code: Your final code submission must provide instructions on how to run the code so that it can be reviewed. Make sure that you have provided a README or equivalent describing the code structure and that the code is documented. In particular, be absolutely that any code that you have used from other sources (not written entirely yourself) is clearly marked to avoid any confusion over plagiarism issues.

Grading Overview

A total of 100 points are possible on each project, allocated as described in this document. Note that a single problem (e.g., inadequate documentation, or a program bug) can cost points in more than one category. For example, if the documentation is not adequate for using a feature, you won't get credit for implementing that feature, nor will you get credit for documenting a feature that doesn't actually work (you may instead lose points for inaccurate documentation). Likewise, if a feature is implemented but cannot be used because the program crashes, you won't get credit for implementing that feature.

15 - Initial Project Plan / SRS / SDS

This initial deliverable for the project will be evaluated by assigning 0 to 1 point to each of the following fifteen bullets based on how well each was completed.

Project Plan

• A management plan. How is your team organized? How is the work divided among team members? How does your team make decisions? How will your team meet and how will it communicate.
• Work breakdown schedule (with > 10 milestones) and project schedule (who will do what).
• Monitoring and reporting: How individual and project progress will be monitored to keep track of who did what and when did they do it?
• A build plan. What is the sequence of steps you will take to build the system? When will each "build" of the system take place
• A rationale for the build plan. Why have you broken the system into these parts, and why have you chosen these particular steps to build the system? What are your risks and your risk reduction strategies?
  

SRS

• Problem statement - Clear and concisely, what is the problem to be solved? The problem, including the task requirements, should be described independently of the solution, the piece of software you will build.
• Description of User - Who are the users? What will they expect? What will they know already? What are their current technology usage patterns?
• Scenarios or Use Cases - Three specific, different, realistic scenarios.
• Detailed Description of Requirements - Expand the problem statement and generalize from the use cases. Include at least 20 specific and measurable requirements. At least 6 should be non-functional requirements.
• Both functional and non-functional requirements should be split between absolutely required and not absolutely required requirements. A reasonable number of absolutely required requirements are identified, well-specified but attainable.

SDS

• A description of the product you intend to build. This should describe the externally visible behavior of your product as precisely as possible, but it should be concise and clear.
• An overall design description. What are the major parts of your system, and how do they fit together?
• System structure is clear.
• Each major subsystem should be explained using a separate static model and dynamic model. All diagrams must be clear and understandable.
• Design Rationale. Why did you chose this particular design? What are the main organizing principles that you used to break your system into parts?

Good Writing

Throughout all sections: Good Writing (From Syllabus) Structure the paper so that the main ideas are clearly accessible. Communicate individual ideas effectively. All spelling and grammar must be standard and correct.

40 - Application Quality

Application quality includes features used by all categories of user. Often this includes administrative users (sysadmins, etc.) in addition to end users. Functionality also includes performance and scalability.

Robustness: 10

This evaluates the extent to which the system can be run with no bugs or errors, such as by failing to install and compile, by behaving differently than specified, or by crashing.

Feature Set: 10

This evaluates the extent to which the system exceeded, met, or fell short of requirements. This considers requirements that were assigned and specified by the team. This evaluates whether essential features were missing and whether extended features were provided.

Ease of Use: 10

This category includes setup (if any) as well as usability for end users and for administrators. It is typically distinct from quality of documentation, but may include presence and usefulness of online help. Usability considerations for administrative users may be quite different than usability considerations for end users; both are considered here. For a prototype, this addresses both existing and planned features.

User Documentation Including Installation and Setup: 10

This evaluates the extent to which documentation clearly and accurately describes, for an end-user, how to accomplish,real-world tasks using the software. This includes how to install and set up the system. The documentation should be clear, accurate, well-written, well-organized, and complete.

45 - Organization, Planning and Technical Documentation

Technical documentation and system organization are evaluated together. Good documentation can make a good design evident, and poor documentation can doom a good design to degradation over time, but good documentation cannot compensate for poor design.

Project Plan: 10 points

This section is evaluated on how effectively the team preforms project planning, how well it followed the plan, how well the team adapts the plan to inevitable hiccups, and whether risks are effectively considered, mitigated, and re-evaluated when plans change.

For full points, a report should clearly indicates who did what, when they did it, and how long everyone spent on each of their tasks (assigned date and completed date, as well as time on task). There should be a clear record of when meetings were held, and what was accomplished and agreed upon at each meeting. A series of continually-updated project plans should show the status of major project milestones and deliverables at regular intervals during the project lifecycle.

Systems Requirements (SRS) - 10 points

This should include all elements described in the initial SRS submission, updated to reflect what was actually built. This should include a clear, complete, and well-organized description of requirements, including a rationale for what is included, what has been deferred to the future, and (as appropriate) what has been excluded. The document should provide a clear user-centric specification as well as a well-defined, precise technical specification. Future developers should be able to use the document for system creation or maintenance.

System Architecture and Design Documentation - 10 points

The software design is communicated as follows:

• Software Architecture: This section should describe the software structure by answering the following questions for a reader:
  • How is the software decomposed into components?
  • How do the components work together to implement the most important application features?
• Why was this particular design chosen? What is the rationale for any key design decisions?
  

• Module Interface Specifications: defines the interface of each component in the design including the services provided, parameters, and the effects of calling each service.

This section should provide an easily understandable specification of the system architecture. Key architectural design decisions should be communicated, along with the rationale for each. The design should satisfy stated design goals.

Technical Documentation - 10 points

This evaluates the technical documentation; that is, the human-readable text that a programmer or human-installer would use to understand (a) how to install the software, (b) software dependencies, (c) required versions of components, (d) how source code files relate to each other, (e) the purpose and inner workings of each source code file, function, if-then statement, and loop, (f) each line of source code, and (g) the purpose, scope, and lifetime of each variable. This includes documents that are separate from the code as well as comments within the code.

Class Presentation - 5 points

Created by A.Hornof, 4/24/18. Adapted from materials created by Stuart Faulk Michal Young.
Removed from Dr. Faulk's evaluation criteria: QA Planning, Developer Logs, and Overall SE and Project Control.