Provide a way to stop locust test when there are problems during on_test_start() event #2875

sfc-gh-tlasica · 2024-08-28T12:13:19Z

sfc-gh-tlasica
Aug 28, 2024

Context:
Before starting benchmark I would like to prepare some data e.g. setup in the database some unique table with data
If this setup fails (for any reason) I do not want to continue, I do not even want to start users
I have used event on_test_start() for this setup (maybe incorrectly?)

Problem:
I do not see how to achieve that.
Raising LocustError() from on_test_start() does not help - seems the exception is swallowed.

Idea
Maybe this event should be able to return boolean to denote if the test should continue?

cyberw · 2024-08-28T13:02:05Z

cyberw
Aug 28, 2024
Maintainer

You should be able to call self.environment.runner.quit() - try it out!

10 replies

whplh Feb 14, 2025

Tried to do somehting similar today.
Here's my take on it:
I want to optionally keep using the ui (just like the stop button would do):

@events.test_start.add_listener
def on_test_start(environment, **kwargs):
    my_max_user_arg = environment.parsed_options.max_user_count
    if environment.runner and environment.runner.target_user_count > my_max_user_arg:
        print("Max users exceeded") # use logging
        if not environment.parsed_options.headless:
            with environment.web_ui.app.test_client() as client:
                client.get("/stop") # Need to top greenlets in ui process. Otherwise weird stuff happens and after some tries worker start spawning "randomly"
                # fyi All code after this line does not execute which is what I want here but maybe not always =)
                print("foo")  
        environment.runner.stop() # not needed in headfull mode, because does not run
        environment.runner.check_stopped() # not needed in headfull mode, because does not run

Here are my reasons: ( in case someone has some feedback):

In case the test_start check fails I want to continue using the ui if it runs and environment.runner.quit() kills the workers in distributed mode and the ui becomes useless
If I don't send the request to /stop, it first seems to work but after a few tries the system starts to try spawning workers from some events earlier...(bug? e.g master-1 | [2025-02-14 17:06:02,179] 26a20ffc0abe/ERROR/root: Max users exceeded master-1 | [2025-02-14 17:06:03,178] 26a20ffc0abe/INFO/locust.runners: Sending spawn jobs of 123 users at 1.00 spawn rate to 2 ready workers)

Anyway, it seems to work for me now.

rragundez Feb 15, 2025

@whplh I tried this but the logs in the UI do not appear, only if I refresh the page and on a second or third try I do see the users being spawned up.

Do you see this as well?

I do not see other users being spawned on subsequent tasks if I use environment.runner.quit(), I still have the problem that the logs are not shown if I am logging/printing the error in the event listener

sfc-gh-tlasica Feb 17, 2025
Author

As I eventually just wanted to stop (skip initialization as well) i ended up with this example (of course skip condition to be replaced with the real cause):

# it should likely go first so that it is executed before on_test_start()
@events.test_start.add_listener
def stop_if_skip_set(environment, **kwargs):
    if os.environ.get("SKIP", "0") == "1":
        logging.error("SKIP locust run")
        sys.exit(0)

which works as below:

locust --headless --host none -f test_perf/dummy_user.py
[2025-02-17 08:20:35,756] TVVGWYQ72D/INFO/locust.main: No run time limit set, use CTRL+C to interrupt
[2025-02-17 08:20:35,756] TVVGWYQ72D/INFO/locust.main: Starting Locust 2.29.1
Type     Name                                                                      # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
--------|------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                     0     0(0.00%) |      0       0       0      0 |    0.00        0.00

[2025-02-17 08:20:35,757] TVVGWYQ72D/ERROR/root: SKIP locust run
[2025-02-17 08:20:35,758] TVVGWYQ72D/INFO/locust.runners: sys.exit(0) called (use log level DEBUG for callstack)

rragundez Feb 17, 2025

That won't work on the Ui right?

sfc-gh-tlasica Feb 17, 2025
Author

That won't work on the Ui right?

Depending how we define working, but indeed - it will not.
UI will open and it will allow you to hit start.
but then it will silently stop.

In general this approach is ruther brutal - it should skip single user not whole run, but I was not able to find anything working reasonably, and we are running single user class in each run.

Locust

Provide a way to stop locust test when there are problems during on_test_start() event #2875

Uh oh!

sfc-gh-tlasica Aug 28, 2024

Replies: 1 comment · 10 replies

Uh oh!

Uh oh!

cyberw Aug 28, 2024 Maintainer

Uh oh!

Uh oh!

whplh Feb 14, 2025

Uh oh!

Uh oh!

rragundez Feb 15, 2025

Uh oh!

Uh oh!

sfc-gh-tlasica Feb 17, 2025 Author

Uh oh!

Uh oh!

rragundez Feb 17, 2025

Uh oh!

Uh oh!

sfc-gh-tlasica Feb 17, 2025 Author

sfc-gh-tlasica
Aug 28, 2024

Replies: 1 comment 10 replies

cyberw
Aug 28, 2024
Maintainer

sfc-gh-tlasica Feb 17, 2025
Author

sfc-gh-tlasica Feb 17, 2025
Author