Package edu.odu.cs.zeil.codegrader

Class TestCase

  • public class TestCase
extends java.lang.Object

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      boolean crashed()
      Did the prior test exit with a non-zero status code?
      void executeTest​(Submission submission, Stage stage)
      Runs the test case using the code in submission.
      int failTest​(Submission submission, java.lang.String message)
      Fail a test without running it.
      java.lang.String getErr()
      Return the captured error stream from a runTest call.
      java.lang.String getOutput()
      Return the captured output from a runTest call.
      TestCaseProperties getProperties()  
      java.lang.String getSubstitutedLaunchCommand​(Submission submission, Stage stage, java.lang.String launch)
      Prepare the launch command to execute a test case, with all substitutions performed.
      int getTime()
      Time in seconds of last runTest call.
      java.lang.String parameterSubstitution​(java.lang.String launchCommandStr, Stage stage, Submission submission)
      Scans a string for shortcuts, replacing by the appropriate string.
      int performTest​(Submission submission, boolean asGold, Stage stage)
      Runs the test case and evaluates the results.
      boolean timedOut()
      Did the prior runTest call go too long?

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • TestCase

        public TestCase​(TestCaseProperties testProperties)
        Create a new test case.
        Parameters:
        testProperties - properties for the test.

    • Method Detail

      • executeTest

        public void executeTest​(Submission submission,
                        Stage stage)
        Runs the test case using the code in submission. Standard out and standard err are captured and available as getOutput() and getErr(). The status code is also available.
        Parameters:
        submission - code to use when running the test case
        stage - staging area

      • getSubstitutedLaunchCommand

        public java.lang.String getSubstitutedLaunchCommand​(Submission submission,
                                                    Stage stage,
                                                    java.lang.String launch)
        Prepare the launch command to execute a test case, with all substitutions performed.
        Parameters:
        submission - submission info
        stage - the stage in which the case will be executed
        launch - the basic launch string
        Returns:
        the launch string with all substitutions

      • performTest

        public int performTest​(Submission submission,
                       boolean asGold,
                       Stage stage)
                throws TestConfigurationError
        Runs the test case and evaluates the results. The actual output, score, and messages are recorded in the recording directory.
        Parameters:
        submission - to evaluate
        asGold - true if the gold version is being run, false if student version is being run.
        stage - stage area in which code has been built
        Returns:
        test case score
        Throws:
        TestConfigurationError

      • failTest

        public int failTest​(Submission submission,
                    java.lang.String message)
             throws TestConfigurationError
        Fail a test without running it.
        Parameters:
        submission - to evaluate
        message - explanation of failure
        Returns:
        test case score
        Throws:
        TestConfigurationError

      • getOutput

        public java.lang.String getOutput()
        Return the captured output from a runTest call.
        Returns:
        captured std out

      • getErr

        public java.lang.String getErr()
        Return the captured error stream from a runTest call.
        Returns:
        captured std err

      • getTime

        public int getTime()
        Time in seconds of last runTest call.
        Returns:
        time

      • timedOut

        public boolean timedOut()
        Did the prior runTest call go too long?
        Returns:
        true iff prior test was killed after exceeding time limit

      • crashed

        public boolean crashed()
        Did the prior test exit with a non-zero status code?
        Returns:
        true iff prior test failed/crashed

      • getProperties

        public TestCaseProperties getProperties()
        Returns:
        the properties for this test case.

      • parameterSubstitution

        public java.lang.String parameterSubstitution​(java.lang.String launchCommandStr,
                                              Stage stage,
                                              Submission submission)
        Scans a string for shortcuts, replacing by the appropriate string. Shortcuts are
        • @P the test command line parameters
        • @S the staging directory
        • @T the test suite directory
        • @t the test case name
        • @R the reporting directory
        A shortcut must be followed by a non-alphabetic character.
        Parameters:
        launchCommandStr - a string describing a command to be run
        stage - the stage where the command will be executed
        submission -
        Returns:
        the launchCommandStr with shortcuts replaced by the appropriate path/value