Lab 6 | CS 61B Spring 2022 Main Course Info Staff Screencasts Beacon GitBugs Resources Piazza Navigation Navigation A. Introduction Writing Tests Lab Structure Useful Videos B. Gitbug C. Unit Testing for Enigma Before You Start Tl;dr of Permutations Alphabet Permutations Writing Unit Tests Grading Using Your Tests for Project 1 D. Acceptance Testing for Enigma Before You Start Generating Acceptance Tests Designing acceptance Tests Grading Error Acceptance Tests E. Submission and Grading Deliverables Submission Lab 6: Unit Testing and Acceptance Testing for Enigma Due Date: Friday 2/25 11:59PM. Navigation A. Introduction B. Gitbug C. Unit Testing for Enigma D. Acceptance Testing for Enigma E. Submission and Grading A. Introduction Pull the files for lab 6 from the skeleton. git fetch shared
git merge shared/lab6 -m "Start lab6"
git push Writing Tests In this lab, we will go over how to test Project 1 (Enigma). There are two components to testing: unit testing and acceptance testing. Unit tests make sure that a function or subset of functions work properly when given a specific input. Acceptance tests ensure that the entire project works properly as a whole. For example, you may write unit tests for the permute method in the Permutation class for Enigma, but you would write an acceptance test to ensure that given a set of configurations and string input, your Enigma machine produces the expected output. Both are important because while individual functions may work properly in an isolated environment, once they start taking in values from other parts of the program, they could break. Here's a very short visual demonstration of why we need acceptance/integration tests. Lab Structure This lab will consist of two parts: Writing unit tests for the Permutation class in Enigma. Writing acceptance tests for Enigma. You will only need to complete one of these two parts to get full credit on the lab. For this lab, we recommend you do the unit testing portion if you are working on the Project 1 Checkpoint and have not yet started writing unit tests for the project, and do the acceptance test portion if you have already made significant headway in completing and testing the project. This lab is designed to help you get started with writing tests for Enigma, but hopefully should not take an extensive amount of time or thoroughness to get full credit for the lab. We highly encourage you to work with a partner to understand Enigma, Permutations, the existing testing methods and utilities, and brainstorming tests and edge cases. HOWEVER, if you are planning on using the tests you write in this lab in your project, you will need to do the actual code-writing by yourself, since code for projects (including tests) must be written by you alone as per our Project Collaboration Policy. Useful Videos Enigma testing video B. Gitbug Since coding assignments in CS61B tend to have various possible implementations, your bug is usually very specific to the code you are writing. As such, it will be impossible to debug in 10 minutes or through a post on Piazza megathread. So in CS61B, we use a system called Gitbugs to give debugging help. As assignments get more involved, it is very important that you use Gitbugs to get debugging help. So, in this lab, we are going to have you practice submitting a Gitbug. In order to receive credits for this part, you have to follow submit a Gitbug following the instructions below: First, please read the Gitbug guide in its entirety to understand how to create a Gitbug request, and what might get your Gitbug request rejected. To access the Gitbug website, go to the course website and click on "Gitbug" on the menu bar as shown below. The Gitbug website looks like the screenshot below. To submit a Gitbug that can be approved by the course staff, you need to fill out this form in detail to describe your bug and show your efforts in debugging. For this lab, select "lab6-gitbug" for the "Assignment" field as follows. Then, read the sample responses on the Gitbug form and fill it out based on a bug you've encountered in any CS61B assignment. For the second last field "Your latest Gradescope submission", please open up the "Lab 6" assignment on Gradescope and copy and paste the URL. For the last field "most recent commit hash", run git log in the terminal in your repo directory. Then copy and paste in the commit hash of you most recent git commit. Lastly, submit your Gitbug. To make sure that your Gitbug was sucessfully submitted, check to make sure you have a lab6-gitbug listed under your Gitbugs at the bottom of the page as the follows. Now, you've learned and practiced how to submit a Gitbug, please use this system to ask for debugging help in future assignments. Submitting a Gitbug is the best way to get help with a bug where the TA needs to look at your code! C. Unit Testing for Enigma You may skip this section if you do not plan on writing unit tests for Permutation in this lab. Before You Start Before you start, please read over the spec for Enigma. You don't need to completely understand everything for this lab, but it will help give context. The part relevant to what you will be testing in this lab is under "Describing Permutations" which will also be described below under "Tl;dr of Permutations". Tl;dr of Permutations Skip this section if you already understand Permutations. Alphabet You are given an abstract class Alphabet which you can assume behaves correctly. In your actual project, it will be a concrete class you will have to implement yourself. Its constructor takes in a string of unique characters that becomes an "alphabet". The index of each character in the string is its index in the Alphabet. For example, if you pass in the String "ABCDEFGHIJKLMNOPQRSTUVWXYZ", 'A' would be index 0, 'Z' would be index 25. The 1-th indexed character would be 'B' and the 24-th indexed character would be 'Y'. Another example, if you pass in the string "MICH", 'M' would be index 0, 'H' would be index 3. The 1-th indexed character would be 'I' and the 2-th indexed character would be 'C'. Permutations You are also given an abstract class Permutation. Like Alphabet, in your actual project, it will be a concrete class you will have to implement yourself. However, for now, it is the class you are writing tests for, and Permutation could either be correct (i.e. follows the spec and docstrings) or buggy. The permutation constructor takes in a String cycles and an Alphabet alphabet. You can think of alphabet as the total set of characters you can work with. cycles must be a String that contains only characters from alphabet, '(', ')', and whitespace between cycles and should follow the format () (), e.g. for the alphabet "HILFNGR", its cycles could be (HIL) (FNGR), (GRL)(HI)(FN), (FHNIGLR), , (HILF) (R), etc. Each grouping of characters enclosed by parenthesis represents one cycle. A Permutation should permute a character at index i of a cycle to the character at index i+1 (wrapping around to the beginning of the cycle if it's the last character) and should invert a character at index i of a cycle to the character at index i-1 (wrapping around to the end of the cycle if it's the first character). A character in the alphabet but not in cycles will map to itself and a character may not show up more than once in cycles. For example, if your alphabet was HILFNGR and cycles was (HIG)(NF) (L), the following are the permutations (x -> y means x permutes to y): H -> I I -> G G -> H N -> F F -> N L -> L R -> R And the inversions would be I -> H, G -> I, etc. (reversing the above arrows). Writing Unit Tests Now let's start writing your own unit tests in PermutationTest. Take a look at the existing files for helper methods and utilities you can use, but you should only have to make additions to PermutationTest. Note that you cannot instantiate Alphabet or Permutation objects directly and must use the helpers in PermutationTests. Your goal for this lab will be to understand a general approach to writing your own unit tests. There are two approaches for writing unit tests: (1) testing per-method, and (2) testing per-case. Testing per-method would be approaching each method one-by-one and writing a test for each kind of input and output you would expect that method to take and give. For example, you could write a test for the permute method and then a test for the invert method. Testing per-case would be approaching the class Permutation as a whole and thinking about what kinds of Permutations could be created and then testing all relevant methods for it. This could be writing a test for a very simple case, e.g. creating a Permutation with one cycle and testing all/most methods on it. In the end, you will want to do a mixture of both to make sure you hit all edge cases. Let's start with a simple test that will test the ability to invert a character. @Test
public void testInvertChar() {
Permutation p = getNewPermutation("(BACD)", getNewAlphabet("ABCD"));
/* TODO: Add additional assert statements here! */
} Recall that the invert function is defined as follows: /** Return the result of applying the inverse of this permutation to C. */
int invert(char c) { ... } Given the permutation above, we know the inverse of 'A' should return 'B'. We can add an assert statements to test this: assertEquals('B', p.invert('A')); Write a few more of your own test cases for this permutation. For example, what should a call to p.invert('B') return? Then continue writing more tests to test expected behavior and edge cases for Permutation. You can think of edge cases by looking at the docstrings in Permutation about each method and rereading the Enigma spec or the section "Tl;dr of Permutations". You may find the provided method checkPerm and JUnit methods such as assertEquals, assertTrue, assertFalse, assertNull, and others useful. If you would like to write a test to make sure a method call throws an exception (not needed for this lab, but will be useful in Enigma), you can write a test in the format: @Test(expected = )
public void test() {
} For the example above, you could ensure that calling p.invert('F') throws an EnigmaException with the following test: @Test(expected = EnigmaException.class)
public void testNotInAlphabet() {
Permutation p = getNewPermutation("(BACD)", getNewAlphabet("ABCD"));
p.invert('F');
} While getting tests to pass actually requires that you completely implement the functions being tested, you don't have to pass them immediately at the time of writing them. Instead, you should create scenarios where you know what is expected to happen. For this lab, you will not be able to actually run your tests on any code - and that's the point. We want you to be able to look a spec and write tests according to that spec, before actually implementing it. This is a common paradigm known as Test-Driven Development (TDD). It's very helpful because it forces you to actually know the expected behavior of the method for a certain input - knowing this will give you an idea of what and how you should implement it. After writing tests in Enigma, don't forget to run the tests as you implement each class in the project to make sure that your method is correct. In addition to writing tests for Alphabet and Permutation as you did for this lab, you should also make sure to write test for Machine and Rotor when working on the project. Grading For this part of the lab, we will be testing your tests against a correct implementation of Permutation, and 12 buggy implementations. For full score, all your tests must pass on the correct implementation of Permutation and at least one test must fail for at least each of 8 of 12 of the buggy implementations. Your tests must pass on the correct implementation to get any credit for this lab. For reference, one solution that was able to get full credit used about 35 lines, but you will likely be writing more (and should be writing more). We highly recommend you try to get your tests to fail on all 12 buggy implementations! Using Your Tests for Project 1 If you would like to copy over PermutationTest.java to your project 1, you will need to copy over PermutationTest.java into proj1/enigma and remove the abstract keyword from the class declaration and from the three abstract methods. You can implement each of the three formerly-abstract methods by having them call the respective constructors to Alphabet and Permutation. D. Acceptance Testing for Enigma Please see Running Enigma in IntelliJ in the project spec for how to run these acceptance tests on your own code. Before You Start Before you start, please read over the spec for Enigma and understand how the Enigma machine works. IMPORTANT: For Windows users, please run the following command: git config --global core.autocrlf true Windows and Unix systems use different line endings in files, and this will ensure that your Windows line endings are always converted to Unix line endings before pushing. Generating Acceptance Tests For Enigma, an acceptance test will test the functionality of the machine as a whole (as opposed to any of the individual parts). Almost all of the autograder tests for Enigma will be acceptance tests. Acceptance tests will be in the proj1/testing/correct or proj1/testing/error folders. We will first talk about tests that go in the proj1/testing/correct folder. Each acceptance test should have a yyy.conf, xxx.in, and xxx.out file. The default default.conf file is: ABCDEFGHIJKLMNOPQRSTUVWXYZ
5 3
I MQ (AELTPHQXRU) (BKNW) (CMOY) (DFG) (IV) (JZ) (S)
II ME (FIXVYOMW) (CDKLHUP) (ESZ) (BJ) (GR) (NT) (A) (Q)
III MV (ABDHPEJT) (CFLVMZOYQIRWUKXSG) (N)
IV MJ (AEPLIYWCOXMRFZBSTGJQNH) (DV) (KU)
V MZ (AVOLDRWFIUQ)(BZKSMNHYC) (EGTJPX)
VI MZM (AJQDVLEOZWIYTS) (CGMNHFUX) (BPRK)
VII MZM (ANOUPFRIMBZTLWKSVEGCJYDHXQ)
VIII MZM (AFLSETWUNDHOZVICQ) (BKJ) (GXY) (MPR)
Beta N (ALBEVFCYODJWUGNMQTZSKPR) (HIX)
Gamma N (AFNIRLBSQWVXGUZDKMTPCOYJHE)
B R (AE) (BN) (CK) (DQ) (FU) (GY) (HW) (IJ) (LO) (MP)
(RX) (SZ) (TV)
C R (AR) (BD) (CO) (EJ) (FN) (GT) (HK) (IV) (LM) (PW)
(QZ) (SX) (UY) The provided trivial.in file is: * B Beta I II III AAAA
HELLO WORLD
* B Beta I II III AAAA
ILBDA AMTAZ and the provided trivial.out file is: ILBDA AMTAZ
HELLO WORLD .conf files are config files that describe the machine and its available rotors and can be shared across multiple tests, the .in files describe the machine's specific configurations for this test and what we are inputting into the machine, and the .out file describes the expected output of the machine. For acceptance testing, you will want to try many different cases, but the most tedious part is figuring out the expected output given some machine configuration and input. Luckily, you have something that can generate the output for you! When writing acceptance tests, you will need to make your own .conf and .in files, but you can use the staff solution. Follow the instruction here to generate the output. Following the instruction, you can run: staff-enigma Note: config file and input file represent paths to those files. and the expected output will be generated and printed. For example, after navigating to the folder with the files default.conf and trivial.in, you can run: staff-enigma default.conf trivial.in and the output will be: ILBDA AMTAZ
HELLO WORLD You can take advantage of this to generate your own test cases! You can alse save the printed output of staff-enigma to a file with a third argument: staff-enigma
