Automated tests¶

Evaluation of weekly exercises uses automatic tests. They have the benefit that you can immediately know, if your program passed the tests or not, and how many points did you earn.

The drawback of automatic tests is that prints shown in the assignments must be exactly correct. Moreover, the error messages given by the automatic tests can be difficult to interpret. Therefore, this section provides you a list of points that you should check if you are wondering the feedback given by the automatic tests.

• Compare carefully the differences, which the automatic testing system shows between your output and the model output. Even a small difference (upper-case/lower-case letter, extra/missing empty space) is meaningful for the testing system.

• Check if there are files or directories generated by the compiler in your central repository (directories beginning with build, files with the suffix .o etc.). If there are such files/directories, remove them, since they may prevent the compiler of the tester working correctly.

  • By no means remove the files via web user interface, but remove them first locally in your own directory, and then push the change into your central repository.
  • To prevent the problem described above in the future, it is highly recommended to create a .gitignore file (see the points 1.8 and 1.8.1 in the course material).

• Uninitialized variables typically produce warnings or errors in program execution that are difficult to interpret. Therefore, please check that you have initialized all the variables. Remember also member variables, i.e. attributes of a class.

• If an automatic test gives no feedback, i.e. it gets stuck, you should suspect an infinite loop in your program. (The stuck can sometimes be due to a server trouble, but in such case, resubmitting after a while is typically effective.)

• If an automatic test gives the warning:

  control reaches end of non-void function...
  

  it means that a function returning a value is lacking a return statement, or the return statement is inside an if structure such that the compiler gives a warning about it.

• Some (especially large) assignments use unit tests. They are tests for checking if a single function works correctly. For them, it is not enough that the program as a whole works correctly, but each function must work correctly.

  • If working of a function is based, for example, on a return value of another function, a unit test may fail due to the other function, not the one, about which the automatic test complains.
  • A unit test may fail, if a function prints something that it should not print, i.e. the program seems to work correctly, but printing happens in a wrong place.