github.com/web-platform-tests/wpt.fyi@v0.0.0-20240530210107-70cf978996f1/webdriver/README.md (about)

     1  # wpt.fyi Webdriver tests
     2  
     3  This directory containers integration tests for webapp/. These tests bring up
     4  the full webserver then use a Golang Webdriver client,
     5  [tebeka/selenium](https://github.com/tebeka/selenium), to load pages in a
     6  browser (Chrome or Firefox) and assert that the webapp behaves as expected.
     7  
     8  To run the tests, from the root `wpt.fyi` directory, run:
     9  
    10      make go_large_test
    11  
    12  You can run just one of Chrome or Firefox via:
    13  
    14      make go_chrome_test
    15      make go_firefox_test
    16  
    17  Note that when running these tests outside of docker (see [Running in
    18  docker](#running-in-docker)), they will use your locally installed browser and
    19  webdriver clients, so it is worth making sure they are the versions you expect.
    20  
    21  ## Debugging
    22  
    23  Debugging the webdriver/ tests can be difficult as they are integration tests
    24  and the problem can occur anywhere from controlling the browser, to the webapp
    25  frontend, to the backend - and other weird bugs inbetween! This section
    26  contains some tips on how to effectively debug them.
    27  
    28  After running one of the above `make` commands at least once, one can directly
    29  run the golang tests via:
    30  
    31  ```
    32  # You can copy GECKODRIVER_PATH out of the make output; it should be installed
    33  # locally under webapp/node_modules/selenium-standalone/...
    34  go test -v -timeout=15m -tags=large ./webdriver -args \
    35      -firefox_path=/usr/bin/firefox \
    36      -geckodriver_path=$GECKODRIVER_PATH \
    37      -chrome_path=/usr/bin/google-chrome \
    38      -chromedriver_path=/usr/bin/chromedriver \
    39      -frame_buffer=true \
    40      -staging=false \
    41      -browser=chrome  # Or firefox
    42  ```
    43  
    44  It is worth comparing this command-line against the Makefile, in case this
    45  documentation becomes out of date.
    46  
    47  ### Running only one test
    48  
    49  If you only need to run one test, you can use the golang test [`-run`
    50  parameter](https://golang.org/pkg/testing/#hdr-Subtests_and_Sub_benchmarks).
    51  For example:
    52  
    53  ```
    54  go test -v -timeout=15m -tags=large ./webdriver \
    55      -run TestProductParam_Order/Order \
    56      -args \
    57      -firefox_path=/usr/bin/firefox \
    58      -geckodriver_path=$GECKODRIVER_PATH \
    59      -chrome_path=/usr/bin/google-chrome \
    60      -chromedriver_path=/usr/bin/chromedriver \
    61      -frame_buffer=true \
    62      -staging=false \
    63      -browser=chrome  # Or firefox
    64  ```
    65  
    66  ### Visual Output
    67  
    68  Many of the tests run some javascript (or click on an element, etc) and expect
    69  to find some resulting change on the page. When that doesn't occur, they
    70  usually just timeout and it can be difficult to know why. One very useful trick
    71  is to enable visual output, so that you can actually see the webpage as the
    72  test runs.
    73  
    74  To do this, set the `frame_buffer` argument to `false`, e.g.:
    75  
    76  ```
    77  go test -v -timeout=15m -tags=large ./webdriver -args \
    78      -firefox_path=/usr/bin/firefox \
    79      -geckodriver_path=$GECKODRIVER_PATH \
    80      -chrome_path=/usr/bin/google-chrome \
    81      -chromedriver_path=/usr/bin/chromedriver \
    82      -frame_buffer=false \
    83      -staging=false \
    84      -browser=chrome  # Or firefox
    85  ```
    86  
    87  ### Verbose webdriver output
    88  
    89  By default, webdriver output is hidden as it is very noisy. You can re-enable
    90  it by passing `-debug=true` to the tests, e.g.:
    91  
    92  ```
    93  go test -v -timeout=15m -tags=large ./webdriver -args \
    94      -firefox_path=/usr/bin/firefox \
    95      -geckodriver_path=$GECKODRIVER_PATH \
    96      -chrome_path=/usr/bin/google-chrome \
    97      -chromedriver_path=/usr/bin/chromedriver \
    98      -frame_buffer=true \
    99      -staging=false \
   100      -browser=chrome \
   101      -debug=true
   102  ```
   103  
   104  Redirecting stderr to stdout and saving it to a log-file is recommended due to
   105  the verbosity of webdriver logs (append `2>&1 | tee my-log.txt` to the above
   106  command).
   107  
   108  ### Running in docker
   109  
   110  Sometimes bugs only occur in a docker-like environment. This can be difficult
   111  to reproduce, but a first step is to run the tests inside of docker. To do
   112  this, first start the docker container in one terminal tab:
   113  
   114  ```
   115  ./util/docker-dev/run.sh
   116  ```
   117  
   118  Then, in another tab, we need to get the instance id of the container, exec
   119  'bash' inside of it, and run our test:
   120  
   121  ```
   122  docker container ls
   123  [ note the CONTAINER ID ]
   124  docker exec -it $CONTAINER_ID bash
   125  user@abce84dd426d:~/wpt.fyi$
   126  [now you can run 'make go_chrome_test', or 'go test ...' directly, etc]
   127  ```
   128  
   129  Note that this maps the host machine's wpt.fyi checkout into docker, so any
   130  code edits you make on the host are reflected in the container and vice-versa.