Skip to content

ci: Add rerun failures package for flaky tests#728

Closed
CagriYonca wants to merge 1 commit into
mainfrom
ci/pytest-rerun
Closed

ci: Add rerun failures package for flaky tests#728
CagriYonca wants to merge 1 commit into
mainfrom
ci/pytest-rerun

Conversation

@CagriYonca
Copy link
Copy Markdown
Contributor

I'm taking the initiative and proposing the pytest-rerunfailures package for flaky tests. Only thing we should do is to add the line just before flaky tests:

    @pytest.mark.flaky(reruns=3, reruns_delay=2)   <-- ONLY THIS
    def test_flaky_test(self):
        ...

Signed-off-by: Cagri Yonca <cagri@ibm.com>
@CagriYonca CagriYonca added do NOT merge Depends on another PR or WIP ci/cd labels May 7, 2025
@CagriYonca CagriYonca self-assigned this May 7, 2025
@CagriYonca
Copy link
Copy Markdown
Contributor Author

CagriYonca commented May 22, 2025

Grpcio fails randomly:

  • FAILED tests/frameworks/test_grpcio.py::TestGRPCIO::test_server_error
    def test_server_error(self) -> None:
        response = None
        with tracer.start_as_current_span("test"):
            try:
                response = self.server_stub.OneQuestionOneErrorResponse(
                    stan_pb2.QuestionRequest(question="Do u error?")
                )
            except Exception:
                pass
    
        assert not get_current_span().is_recording()
        assert not response
    
        spans = self.recorder.queued_spans()
>       assert len(spans) == 4
E       assert 3 == 4

https://app.circleci.com/pipelines/github/instana/python-sensor/3997/workflows/8223d7a0-aaa6-4c73-b322-6ea4c97f4ae5/jobs/30757/parallel-runs/0/steps/0-105

@CagriYonca
Copy link
Copy Markdown
Contributor Author

CagriYonca commented Jun 18, 2025

Tornado fails randomly:

  • tests/frameworks/test_tornado_server.py::TestTornadoServer::test_get_with_params_to_scrub
    def test_get_with_params_to_scrub(self) -> None:
        async def test():
            with tracer.start_as_current_span("test"):
                async with aiohttp.ClientSession() as session:
                    return await self.fetch(session, testenv["tornado_server"], params={"secret": "yeah"})
    
        response = tornado.ioloop.IOLoop.current().run_sync(test)
    
        spans = self.recorder.queued_spans()
>       assert len(spans) == 3
E       AssertionError: assert 2 == 3

https://app.circleci.com/pipelines/github/instana/python-sensor/4071/workflows/f52e3efe-42c0-4972-8a99-291b961743e1/jobs/31732/parallel-runs/0/steps/0-105

@CagriYonca CagriYonca closed this Sep 1, 2025
@CagriYonca
Copy link
Copy Markdown
Contributor Author

CagriYonca commented Sep 1, 2025

Not needed anymore

@CagriYonca CagriYonca deleted the ci/pytest-rerun branch November 3, 2025 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd do NOT merge Depends on another PR or WIP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant