Class TestSuite

  • All Implemented Interfaces:
    java.lang.Iterable<TestCase>

    public class TestSuite
    extends java.lang.Object
    implements java.lang.Iterable<TestCase>
    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  TestSuite.BuildResult  
    • Constructor Summary

      Constructors 
      Constructor Description
      TestSuite​(Assignment asst)
      Create a new test suite.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void addJavaSrcDir​(java.lang.String relativePath)
      Adds a directory that is expected to hold Java source code.
      void clearTag​(java.lang.String tagName)
      Clear the named tag and remove any unperformed test cases with that tag name as its kind from the queue of tests to be performed.
      void clearTags()
      Clear all tags and clear the queue of tests to be performed.
      int computeDaysLate​(Submission submission)
      If both a due date and a submission date are available, compute how many days late this submission is.
      java.lang.String getAssignmentName()
      Get a name for the assignment.
      java.lang.String getLockDate​(Submission sub)
      Attempt to determine when this submission was turned in.
      int getNumTestsToPerform()  
      java.lang.String getSubmissionDate​(Submission sub)
      Attempt to determine when this submission was turned in.
      boolean isTagActive​(java.lang.String tagName)  
      java.util.Iterator<TestCase> iterator()
      Provides access to the list of test cases in this suite.
      boolean needsRegrading​(java.nio.file.Path recordAt, Submission submission)
      Check to see if a submission needs to be (re)graded.
      java.time.LocalDateTime parseDateTime​(java.lang.String dateTimeString)
      Flexible parsing of dates and time.
      void performTests()
      Perform all selected tests for all selected submissions.
      void processThisSubmission​(Submission submission)
      Process a submission.
      void setDueDate​(java.lang.String date)
      Set the due date for this suite.
      void setLaunch​(java.lang.String launcher)
      Set the default launch command for all test cases in this suite.
      void setSelectedSubmissions​(java.util.List<java.lang.String> submissionList)
      Limits the set of submissions that will actually be run by performTests.
      void setSelectedTests​(java.util.List<java.lang.String> testList)
      Limits the set of tests that will actually be run by performTests.
      void setSubmissionDateByGit​(boolean setByGit)
      Submission dates will be read from a file.
      void setSubmissionDateIn​(java.lang.String filePath)
      Submission dates will be read from a file.
      void setSubmissionDateMod​(java.lang.String filePath)
      Submission dates will be determined by the last modification date of a file.
      void setTag​(java.lang.String tagName)
      Set the named tag and queue up any unperformed test cases with that tag name as its kind.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
      • Methods inherited from interface java.lang.Iterable

        forEach, spliterator
    • Constructor Detail

      • TestSuite

        public TestSuite​(Assignment asst)
        Create a new test suite.
        Parameters:
        asst - the assignment that this test suite is intended to assess.
    • Method Detail

      • setLaunch

        public void setLaunch​(java.lang.String launcher)
        Set the default launch command for all test cases in this suite.
        Parameters:
        launcher - launcher command
      • setDueDate

        public void setDueDate​(java.lang.String date)
        Set the due date for this suite.
        Parameters:
        date - the new date
      • setSelectedTests

        public void setSelectedTests​(java.util.List<java.lang.String> testList)
        Limits the set of tests that will actually be run by performTests.
        Parameters:
        testList - list of test case names
      • setSelectedSubmissions

        public void setSelectedSubmissions​(java.util.List<java.lang.String> submissionList)
        Limits the set of submissions that will actually be run by performTests.
        Parameters:
        submissionList - list of submission names
      • setSubmissionDateMod

        public void setSubmissionDateMod​(java.lang.String filePath)
        Submission dates will be determined by the last modification date of a file.
        Parameters:
        filePath - path to submission file
      • setSubmissionDateIn

        public void setSubmissionDateIn​(java.lang.String filePath)
        Submission dates will be read from a file.
        Parameters:
        filePath - path to submission file
      • setSubmissionDateByGit

        public void setSubmissionDateByGit​(boolean setByGit)
        Submission dates will be read from a file.
        Parameters:
        setByGit - true means that if a submission directory is a git repository, the last commit date will be used as the submission date.
      • performTests

        public void performTests()
        Perform all selected tests for all selected submissions. The entire sequence is performed: 1) build gold version (if available) for each submission, 2) Copy the test suite into the reporting area. 3) set up submission code in staging area 4) build code in staging area for all selected test cases 5) if a gold version is available, run the test on it and collect the expected output and expected time limit. 6) run and evaluate the test on the submission , putting results in the recording area 7) Prepare summary reports of test results
      • processThisSubmission

        public void processThisSubmission​(Submission submission)
        Process a submission. 1) Copy the test suite into the reporting area. 2) set up submission code in staging area 3) build code in staging area for all selected test cases run and evaluate the test case. 4) Prepare summary reports of test results
        Parameters:
        submission - the submission to process
      • needsRegrading

        public boolean needsRegrading​(java.nio.file.Path recordAt,
                                      Submission submission)
        Check to see if a submission needs to be (re)graded.
        Parameters:
        recordAt - directory where grade results should be recorded.
        submission - the submission to check
        Returns:
        true if submission should be regraded.
      • computeDaysLate

        public int computeDaysLate​(Submission submission)
        If both a due date and a submission date are available, compute how many days late this submission is.
        Parameters:
        submission - a submission
        Returns:
        number of days late (0 if on time)
      • parseDateTime

        public java.time.LocalDateTime parseDateTime​(java.lang.String dateTimeString)
                                              throws java.time.format.DateTimeParseException
        Flexible parsing of dates and time.
        Parameters:
        dateTimeString - a date or date and time
        Returns:
        equivalent date-time
        Throws:
        java.time.format.DateTimeParseException
      • getAssignmentName

        public java.lang.String getAssignmentName()
        Get a name for the assignment. If not given in the .yaml file, a default value is taken from the directory name containing the test suite.
        Returns:
        an assignment name
      • addJavaSrcDir

        public void addJavaSrcDir​(java.lang.String relativePath)
        Adds a directory that is expected to hold Java source code. An assignment can hold multiple such source roots (e.g. the Maven & Gradle conventions place Java source code in src/main/java and src/test/java). *.java files that are found out-of-place according to their package declarations will be eventually moved into the appropriate location within the file tree of the first such source directory added.
        Parameters:
        relativePath - a Java source directory root.
      • iterator

        public java.util.Iterator<TestCase> iterator()
        Provides access to the list of test cases in this suite.
        Specified by:
        iterator in interface java.lang.Iterable<TestCase>
      • getSubmissionDate

        public java.lang.String getSubmissionDate​(Submission sub)
        Attempt to determine when this submission was turned in.
        Parameters:
        sub - the submission
        Returns:
        a string representing a date and/or time, or "".
      • getLockDate

        public java.lang.String getLockDate​(Submission sub)
        Attempt to determine when this submission was turned in.
        Parameters:
        sub - the submission
        Returns:
        a string representing a date and/or time, or "".
      • getNumTestsToPerform

        public int getNumTestsToPerform()
        Returns:
        the number of test cases currently queued up to be run.
      • isTagActive

        public boolean isTagActive​(java.lang.String tagName)
        Parameters:
        tagName - tag name
        Returns:
        true iff this tag has been set
      • setTag

        public void setTag​(java.lang.String tagName)
        Set the named tag and queue up any unperformed test cases with that tag name as its kind.
        Parameters:
        tagName - a tag name
      • clearTag

        public void clearTag​(java.lang.String tagName)
        Clear the named tag and remove any unperformed test cases with that tag name as its kind from the queue of tests to be performed.
        Parameters:
        tagName - a tag name
      • clearTags

        public void clearTags()
        Clear all tags and clear the queue of tests to be performed.