Package edu.odu.cs.zeil.codegrader
Class TestSuite
- java.lang.Object
-
- edu.odu.cs.zeil.codegrader.TestSuite
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
TestSuite.BuildResult
-
Constructor Summary
Constructors Constructor Description TestSuite(Assignment asst)
Create a new test suite.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
addJavaSrcDir(java.lang.String relativePath)
Adds a directory that is expected to hold Java source code.void
clearTag(java.lang.String tagName)
Clear the named tag and remove any unperformed test cases with that tag name as its kind from the queue of tests to be performed.void
clearTags()
Clear all tags and clear the queue of tests to be performed.int
computeDaysLate(Submission submission)
If both a due date and a submission date are available, compute how many days late this submission is.java.lang.String
getAssignmentName()
Get a name for the assignment.java.lang.String
getLockDate(Submission sub)
Attempt to determine when this submission was turned in.int
getNumTestsToPerform()
java.lang.String
getSubmissionDate(Submission sub)
Attempt to determine when this submission was turned in.boolean
isTagActive(java.lang.String tagName)
java.util.Iterator<TestCase>
iterator()
Provides access to the list of test cases in this suite.boolean
needsRegrading(java.nio.file.Path recordAt, Submission submission)
Check to see if a submission needs to be (re)graded.java.time.LocalDateTime
parseDateTime(java.lang.String dateTimeString)
Flexible parsing of dates and time.void
performTests()
Perform all selected tests for all selected submissions.void
processThisSubmission(Submission submission)
Process a submission.void
setDueDate(java.lang.String date)
Set the due date for this suite.void
setLaunch(java.lang.String launcher)
Set the default launch command for all test cases in this suite.void
setSelectedSubmissions(java.util.List<java.lang.String> submissionList)
Limits the set of submissions that will actually be run by performTests.void
setSelectedTests(java.util.List<java.lang.String> testList)
Limits the set of tests that will actually be run by performTests.void
setSubmissionDateByGit(boolean setByGit)
Submission dates will be read from a file.void
setSubmissionDateIn(java.lang.String filePath)
Submission dates will be read from a file.void
setSubmissionDateMod(java.lang.String filePath)
Submission dates will be determined by the last modification date of a file.void
setTag(java.lang.String tagName)
Set the named tag and queue up any unperformed test cases with that tag name as its kind.
-
-
-
Constructor Detail
-
TestSuite
public TestSuite(Assignment asst)
Create a new test suite.- Parameters:
asst
- the assignment that this test suite is intended to assess.
-
-
Method Detail
-
setLaunch
public void setLaunch(java.lang.String launcher)
Set the default launch command for all test cases in this suite.- Parameters:
launcher
- launcher command
-
setDueDate
public void setDueDate(java.lang.String date)
Set the due date for this suite.- Parameters:
date
- the new date
-
setSelectedTests
public void setSelectedTests(java.util.List<java.lang.String> testList)
Limits the set of tests that will actually be run by performTests.- Parameters:
testList
- list of test case names
-
setSelectedSubmissions
public void setSelectedSubmissions(java.util.List<java.lang.String> submissionList)
Limits the set of submissions that will actually be run by performTests.- Parameters:
submissionList
- list of submission names
-
setSubmissionDateMod
public void setSubmissionDateMod(java.lang.String filePath)
Submission dates will be determined by the last modification date of a file.- Parameters:
filePath
- path to submission file
-
setSubmissionDateIn
public void setSubmissionDateIn(java.lang.String filePath)
Submission dates will be read from a file.- Parameters:
filePath
- path to submission file
-
setSubmissionDateByGit
public void setSubmissionDateByGit(boolean setByGit)
Submission dates will be read from a file.- Parameters:
setByGit
- true means that if a submission directory is a git repository, the last commit date will be used as the submission date.
-
performTests
public void performTests()
Perform all selected tests for all selected submissions. The entire sequence is performed: 1) build gold version (if available) for each submission, 2) Copy the test suite into the reporting area. 3) set up submission code in staging area 4) build code in staging area for all selected test cases 5) if a gold version is available, run the test on it and collect the expected output and expected time limit. 6) run and evaluate the test on the submission , putting results in the recording area 7) Prepare summary reports of test results
-
processThisSubmission
public void processThisSubmission(Submission submission)
Process a submission. 1) Copy the test suite into the reporting area. 2) set up submission code in staging area 3) build code in staging area for all selected test cases run and evaluate the test case. 4) Prepare summary reports of test results- Parameters:
submission
- the submission to process
-
needsRegrading
public boolean needsRegrading(java.nio.file.Path recordAt, Submission submission)
Check to see if a submission needs to be (re)graded.- Parameters:
recordAt
- directory where grade results should be recorded.submission
- the submission to check- Returns:
- true if submission should be regraded.
-
computeDaysLate
public int computeDaysLate(Submission submission)
If both a due date and a submission date are available, compute how many days late this submission is.- Parameters:
submission
- a submission- Returns:
- number of days late (0 if on time)
-
parseDateTime
public java.time.LocalDateTime parseDateTime(java.lang.String dateTimeString) throws java.time.format.DateTimeParseException
Flexible parsing of dates and time.- Parameters:
dateTimeString
- a date or date and time- Returns:
- equivalent date-time
- Throws:
java.time.format.DateTimeParseException
-
getAssignmentName
public java.lang.String getAssignmentName()
Get a name for the assignment. If not given in the .yaml file, a default value is taken from the directory name containing the test suite.- Returns:
- an assignment name
-
addJavaSrcDir
public void addJavaSrcDir(java.lang.String relativePath)
Adds a directory that is expected to hold Java source code. An assignment can hold multiple such source roots (e.g. the Maven & Gradle conventions place Java source code in src/main/java and src/test/java). *.java files that are found out-of-place according to their package declarations will be eventually moved into the appropriate location within the file tree of the first such source directory added.- Parameters:
relativePath
- a Java source directory root.
-
iterator
public java.util.Iterator<TestCase> iterator()
Provides access to the list of test cases in this suite.- Specified by:
iterator
in interfacejava.lang.Iterable<TestCase>
-
getSubmissionDate
public java.lang.String getSubmissionDate(Submission sub)
Attempt to determine when this submission was turned in.- Parameters:
sub
- the submission- Returns:
- a string representing a date and/or time, or "".
-
getLockDate
public java.lang.String getLockDate(Submission sub)
Attempt to determine when this submission was turned in.- Parameters:
sub
- the submission- Returns:
- a string representing a date and/or time, or "".
-
getNumTestsToPerform
public int getNumTestsToPerform()
- Returns:
- the number of test cases currently queued up to be run.
-
isTagActive
public boolean isTagActive(java.lang.String tagName)
- Parameters:
tagName
- tag name- Returns:
- true iff this tag has been set
-
setTag
public void setTag(java.lang.String tagName)
Set the named tag and queue up any unperformed test cases with that tag name as its kind.- Parameters:
tagName
- a tag name
-
clearTag
public void clearTag(java.lang.String tagName)
Clear the named tag and remove any unperformed test cases with that tag name as its kind from the queue of tests to be performed.- Parameters:
tagName
- a tag name
-
clearTags
public void clearTags()
Clear all tags and clear the queue of tests to be performed.
-
-