Trouble Encountering 403 Forbidden Error When Using DebugBench with LeetCode #6
Description
Hello!
I encountered the same issue 4, when i try to test the generated code through LeetCodeTester. I receive the following error:
Traceback (most recent call last):
File "d:\MyProject\Work\code_eval_syh\DebugBench-main\evaluation\leetcode_oj\leetcode_tester.py", line 55, in
print(tester.test(code, task_id, "python"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\MyProject\Work\code_eval_syh\DebugBench-main\evaluation\leetcode_oj\leetcode_tester.py", line 42, in test
status, reward, done, submission_result = self.env.step(sub)
^^^^^^^^^^^^^^^^^^
File "d:\MyProject\Work\code_eval_syh\DebugBench-main\evaluation\leetcode_oj\leetcode_env\environment.py", line 58, in step
submission_result = self.__send_submission(action)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\MyProject\Work\code_eval_syh\DebugBench-main\evaluation\leetcode_oj\leetcode_env\environment.py", line 86, in __send_submission
submission_id = self.api_instance.problems_problem_submit_post(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\anaconda3\envs\private\Lib\site-packages\leetcode\api\default_api.py", line 385, in problems_problem_submit_post
(data) = self.problems_problem_submit_post_with_http_info(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\anaconda3\envs\private\Lib\site-packages\leetcode\api\default_api.py", line 465, in problems_problem_submit_post_with_http_info
return self.api_client.call_api(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\anaconda3\envs\private\Lib\site-packages\leetcode\api_client.py", line 345, in call_api
return self.__call_api(
^^^^^^^^^^^^^^^^
File "D:\anaconda3\envs\private\Lib\site-packages\leetcode\api_client.py", line 159, in __call_api
response_data = self.request(
^^^^^^^^^^^^^
File "D:\anaconda3\envs\private\Lib\site-packages\leetcode\api_client.py", line 422, in request
return self.rest_client.POST(
^^^^^^^^^^^^^^^^^^^^^^
File "D:\anaconda3\envs\private\Lib\site-packages\leetcode\rest.py", line 338, in POST
return self.request(
^^^^^^^^^^^^^
File "D:\anaconda3\envs\private\Lib\site-packages\leetcode\rest.py", line 250, in request
raise ApiException(http_resp=r)
leetcode.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Date': 'Fri, 25 Apr 2025 07:56:10 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '8806', 'Connection': 'close', 'accept-ch': 'Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA', 'cf-mitigated': 'challenge', 'critical-ch': 'Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA', 'cross-origin-embedder-policy': 'require-corp', 'cross-origin-opener-policy': 'same-origin', 'cross-origin-resource-policy': 'same-origin', 'origin-agent-cluster': '?1', 'permissions-policy': 'accelerometer=(),autoplay=(),browsing-topics=(),camera=(),clipboard-read=(),clipboard-write=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()', 'referrer-policy': 'same-origin', 'server-timing': 'chlray;desc="935c4843dba0a064"', 'x-content-type-options': 'nosniff', 'x-frame-options': 'SAMEORIGIN', 'Cache-Control': 'private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'Expires': 'Thu, 01 Jan 1970 00:00:01 GMT', 'Vary': 'Accept-Encoding', 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains; preload', 'Server': 'cloudflare', 'CF-RAY': '935c4843dba0a064-FRA'})
HTTP response body: <title>Just a moment...</title><style>*{box-sizing:border-box;margin:0;padding:0}html{line-height:1.15;-webkit-text-size-adjust:100%;color:#313131;font-family:system-ui,-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color Emoji}body{display:flex;flex-direction:column;height:100vh;min-height:100vh}.main-content{margin:8rem auto;max-width:60rem;padding-left:1.5rem}media (width <= 720px){.main-content{margin-top:4rem}}.h2{font-size:1.5rem;font-weight:500;line-height:2.25rem}media (width <= 720px){.h2{font-size:1.25rem;line-height:1.5rem}}#challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+);background-repeat:no-repeat;background-size:contain;padding-left:34px}media (prefers-color-scheme:dark){body{background-color:#222;color:#d9d9d9}}</style><script>(function(){window._cf_chl_opt={cvId: '3',cZone: "leetcode.com",cType: 'managed',cRay: '935c4843dba0a064',cH: '1BV.oY6Nh5qjjLSDHjFPeWSlUdwUC40qXgSbotkpNZ0-1745567770-1.2.1.1-i1MpcLLTUTYhCAyEgSRWGgPAG_vNabZQuL0CeAuI09U16JaVY.F.aJ.aK8GvUJgk',cUPMDTk: "/problems/make-number-of-distinct-characters-equal/submit/?__cf_chl_tk=L2H4yk_YZWKgKd37bBskMEnah77zT1030q6tEccPHt4-1745567770-1.0.1.1-YtVHh1zBQFdzKF1PaAC16DYRKtVcS33Ls8gSgZar6g0",cFPWv: 'g',cITimeS: '1745567770',cTplC: 0,cTplV: 5,cTplB: 'cf',cK: "unsupported_browser_beacon",fa: "/problems/make-number-of-distinct-characters-equal/submit/?__cf_chl_f_tk=L2H4yk_YZWKgKd37bBskMEnah77zT1030q6tEccPHt4-1745567770-1.0.1.1-YtVHh1zBQFdzKF1PaAC16DYRKtVcS33Ls8gSgZar6g0",md: "lBk_P9YghxB1PnDElJUxwcV0oi5DIQz2W6HC2rg40oI-1745567770-1.2.1.1-gk1c3ZYyThmzwZKxvKhQHbYOsBSHAXa5lyX2Fjnbm.CvEcanX_ONzbBiyJorBSUA_Qc_izDFQEwoDlkAR78UbV4QJa6mAFwzPcBqywqd5fPKmTDZ_7FFF1h6x9CoGm3Q56k.QZD944bVUt1ZliTyItHhtt9VRh2x3hv.qixdL7J_uoG2AmSdIUwUsw77vEUEtVFr3RkFCCET6yjkevfL2WHwV91n59EIjMfAlt3C6VMWM41oHrlHr0t4qJMv_24UU972a20Myif_gixtiaaNJ_IpaXsdAFM6D7YQ8WTti2emf.1uNFchYtmmZ.7tAjfuUadsYDGMMiw9e20DBtVcio7faXMnIUmTK4eAWo5eZfEtNERK7EY4Z4bRUWOO4OO0m.7AWpnflesP0PMopRpaYcvUoq7Aaf2Ref9wbpj9YHEYDgkzjXIFV0acyPYgyvr3o.kdvMR_SPeTT.xENhf_TlOZrA_oHL.hiOWirjXn10TdjleBVMFC_EAlSbxGAn3axh35LOToDkc_rMntxVJ35lkkK3JJVYqHS3fpHYNpCNlitAf9ITPuBvRv1uC_Sz7BZYqTY8ygIDPEMzMvOAG3L9mSk1zy236T85Adtv02m2Ukt7pYJW99QxzWejp8kcVJzT1KSpSvO7aFGoyCEotDKryfA1dYm9NDzOvfxw.OJqQHD0MzCyh4AvlemtHL3gq0mwGx88v5DKttOlw8mEjGOvpe_2c44a9gJQaGQ9DAFREqa2KQO7ld6Yj0yGEHYCrhPAqsOjRDAta7_rnILXCAgTLU0OE6w7pMVKr0FrQVOX0aophEjuLKCq_Zi1XcH9rWaMNwyCAPbzt.9dFERmGXT9HZjbKHvhzFs3elbgfZPZCmivNCDkt0thJG798otg9_RgZGIV_SgK_5wacNN3CPxKh42d5jOiMHR0KodIRZl2wlntSPkyu2rONZKkku6LA1ohgcz4Q8PJ8ip2GMgdNuBbaTyxaNtEg7gYl2J9nXBee4.b.FPEY81W.CrqJfhOJ2vc9470L_fh45o.RDwHehEvMX3nHuRqwAiwo9OLRRM3_7Q5hC1USO6qU8eDEa5ABHonfsAfumrifu9XcHdYCJXiOCeAFITInB.PnoSKwnc_HlsOYPWzAi4IM8TNe3Ztro",mdrd: "2jHI_iiZeqhsB40HcCvQ.rJSHsqTcXZjTLueVjg7ROs-1745567770-1.2.1.1-AnLuVvhP_I1CMnp5GIbM7rdUrqv37wf1DbkAIMUN2HbZXLlLFz8kg2P02pmxG.TNW_Y09SKSa1cYbkko_5NqCyrecJOvQlOlgektYsK6ZQUPAXU8Ku8Fe9QKpQB27WjlMKlCbuBSLetWBDSb.fVeJrs2ngLqwZOjYFwDcWzNgFtnswwCPFMEZ92F26AIMIZIrjTWfOkA5XtgUNRDRjEE2iMPWh.PUpviJ2nhQ9nsM9MOp5Gf.9IGP_pmpdLhKjofrDO0T7_yxoyLBJ8sjw8AZgov9mnH8.jhgrM2LZ6IVcqtuj57ToT0pLLsRdLGfrmBCumsGxnV7Pf.pq6n4kJN.qycRci9qhRoDR3kFjo5TwrKC717vooilFC4VHFuGCExdaYuHAHue6es4f2owqfjv0mI6KHIgaPRgcYnHCy5t9lOED.6MzmyDaKjMhCSTGDy81oc.zXqCFcWUtYzfEXZE9QiLJHZs.Epypl.vzt7L8ob2By6VePVAyAc_gpmGzfK82FpcRD2TiVm1_kTJZ5gXasTaTROf821QlDm2zEZvNfMWKv_F_zzTXByQ3bIRyLgSPQEMINVDyOBb3.rR_pASyt1KieJHxgn6.uduNCK46QqkCCPDYDIXYY6IgAFeNREw9apbnmsxEF4iUSvzsfNKxSnfDFfgIGxC9YYzSmXAXZgQuN4UhE0gkNzCsdjj0gy8CxqMkkmd0Qy9rLhrLRs1mVBeNqqv3xgWjAh2cdUy_TgU_4jthxtQwSmCOKz1jTpwvvZUduTarK.fvqsx7i_rR9OaVKTbuVbhG0.Mc2S6vXBFuYe6PvJa.WOUq5wXT1jAFOCZ.k0AfULNv98ESZDEBqom.1AOUFCnFlmw8r2_9mDR1ptHQvVvEc5QJgs1PPcrE4LHAGKNLHBDm3q4q7QBiWaqd0uBV6iVefNBTA.Nuu3yzvIthEcgvCRJp2XQQu2LfZtKWsQV9J.zMPmAFPr7s8U3jye9iu_hCIxm_omnt45ZhAoYsRvjdYlg36CWx6El7h08Tm5gMtdlRZzeYIt.ZCQ1EMNVXuhKgTQD3ZabpQNW1z0a8bie4k7AzFyklPaBEEOKWOcKRV1yRX3QxfcfpVEGziy.u4vs0kYT1tpKHpXjJMWgDkrT1ySU.cunLNF_ZozI1Kmv4HvNtXEDMNvW2ge9tpQdzYYiau8TO2I3wJzFwbsEgtDjoVVhSeQHRu1us_Bhj1K7UUqJwb36wGfZ49pRv2QOl8wIUplxmUrZ5H0R8y.G8BmvAW3h8JZ_H9hKH0p87IztLIIBfRCBgef9THfNTumSV4QXXm7QCAXxPoEvYzFT1hddbupE.lpPbu3Qt4zNNkXQgOIw_4iEo.MYFVXhWH_LWSbPmLQFDY93wWZxRXQ0Iav_6AwkFXiP6MGcHIa6rZ9niIXm9pg0.2zAwhXjQ5IeXZVHmMGMajublxAY00H6xZdbWlYZE0MNSiSpp40cmvZy55QIKU5xgp5voTkxeRH0AP6a6nxaCsjIQFkCAsOBupUxHes3zDpt1n9.4ERICzkI2NJZjqPSKroetarceNmKw.mN6Y43gWtECL2DYocqY7joqObFXjFxlobpC1CODpezcq0bxl59OIL1CUKpahdeB3iRnNbPMs5laGMxLIghF1gdekdvjVn6ec5jc3YgMZ2ndfyqGzZJaf3iNup.NLxg0M8nFJX0TCCqFVB1vHT04YO0QowLBxQtdrHe_l3VzokiZbhukSBA6DcjrOBdqMuGcChQ1.6VICwbtDB2jgBHzwDlmVGNlTiGZtybD.xqEscdJhm2b7IppkM37sjgci1UvMIKQOy9Q4JEI4SM9vGNSaUx5mH632fhFQgN6yys6K.FLst01KYouyOr8hKVaQbPFhgEWdcyodzudiLk.z5Yez37pr1cfhqPx2BL8htUYJEP2E_DDh4b3RggP1o3xzzVmRPyOiBNEQ8gfmUah_17W5ubX0HwXl4IXYD8yRoM_EMGBg2xBxy7AO5Rcc2GneiAVrJ8uM1OI1GJtMEB25rNrD9Xszb9IJy7aDvaIVNKooxkbxaz36D0iyAWEMnpK1mrngCFYHX1cVwXNjrysh0o1AfWn7hDnSTMkvdO5VajReHDNr2pYXoLutb70CLjlzulgnXboSNqzgUrS_PvIXc.Koe.JJA2zFEDVmjACwmA_cmYQML5ziYqMwrnQTJfZFODbCgvSJ34aXFp0V5jscUZZI_qdcmV0JIwKZnKNy78W1oZKScd31gF.pWRcTkfLL7uo38ua0KXy7RUOvbXEY5ETPOHgfBJ1gkhU7aXw9kuHZHrr0_OgAoouEAmj8DVZ1B9S4aWAbJBZob2NilKB_Dos1qO_gyondAHUlxzTu3PyVDj0VkWO__RdqPoa8sIo3oQQD2spPkduvymsrFfolxhulDL.WUs6ATptHaap.505kPqV3fonf4MtXWNS7NUH2rzh7jIq.ouvQMf8xduaQzYNh9QFCPivtgywhvqVn8ZkF.OAlmKoA3up.Lo1A0SWKuTZ74MCzZm2FS1dtNokGrbGWPDpe.a4_dGJJSctS_OhGlndFg9j9m15vv9bY1_FGyLo366YDUH1_RdTESSfQzuBGVHNQKaXsHNhdgiY6j_Cu8701zhUaKJDdhuuQgnRQGz3on8PrXQqjU.QGGuAnHUEMcQGRcqyG8ULIz2UkjTFv6HuwZRvBrg4dtH1ycpFppzDcbI6u5at8fDjFCmWNQY0VGQtBhsGKwzHfaAsoES5JtXvEPSkNRzzXuLnOkKFLME4O.w1QHMTEorthHkzKdSKDu09ieiL4fm3A7v8gu3r37jaxprMAhK58mXTAIm8gX_bek2PbTzAc30xgs95078ZtxIZ4Is5Gl2E3lZ3_VmmGaRfVCwFSim6wSUfq9TJs6ZwDehpovgyuYxOquBsDTn_Ul8hTWhKNYFwUkw3bKYHWiFJxgHQIN6Cngn6oYM_MNnsbD83gekarlR9WgPZd1QGx_v4lxZihhIg66DvuhTOubA4H7o_d9Q5TLUKgN0iUjav6FkLRem0LM24X_U9r6MZCGJnYSWX3H.Y0bSZvww1eNdW4.5HH_f47d.ywDVrbNUl8ozgJHB7phcn9gpbzS4ltMWqFvdOIA8XZAwU2DG3JX8hFeX2PaPXk7MP46FwKJyMQVzhoLwcPLyhAp2F0IUAqwr10HwcXdk3Q4rPVvhJ9z7Fb8bEV.IVZNsq6EgIAZcF3YCnuq49JiE3Ce37b5j8lSiai81kc9gNcUz6DI_C06TpqTe5terGcU0t9cCx6grOrg9F5FQtnBWwoYy3UQt2ABJdhj18Nw2HtbE1VjhtbwtYTDUuJPwMdS5i9gwR70SRnz5eqLZzUppwGUCUdTcC2EZ3A_MgQl.JVVI2l46caBO2TxIflDHJs7E2Cz_2WHjM9hygSg6E5ZId4vrncQYJ80g.bKrC5OURtzVk29T.z9kRthMfQLnmfCfaf0a5KjkMxFQMJztqPqrKw9XTrZbLxx9ehiW_NHuv1CmQOR_gTQWip3.bXoRP4qD4vSSw7yP5yu7Oic2MgvfWqLHk9DAotj.bnPRiNXonog2Do_vF_WY0iwk.FztBKk37UFB3hLqi1K428yU4pLsF.z7PWK2bNx.045dYWIKbAGvqfzKtq.6iKrKQlGczYhmtYiDREZx1a.Ic7f5KXkAr5oq9n9.JcXqVn0QeVzpqeO.5lB0JW.3K90cLQkbPNZxLJKh.6mMxniTlTwUF93gJ1LMAUE4n955WOGsvzwKTxYva3O.70Ej9.hZ0KRkqBwrqO8l7_Rskl8hQCSDvSPbX6gDKxDqE5.5iZgOW56n5oi9l.1vlr2Ta_6zn8hcTc41g"};var cpo = document.createElement('script');cpo.src = '/cdn-cgi/challenge-platform/h/g/orchestrate/chl_page/v1?ray=935c4843dba0a064';window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null, "/problems/make-number-of-distinct-characters-equal/submit/?__cf_chl_rt_tk=L2H4yk_YZWKgKd37bBskMEnah77zT1030q6tEccPHt4-1745567770-1.0.1.1-YtVHh1zBQFdzKF1PaAC16DYRKtVcS33Ls8gSgZar6g0" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null, null, ogU);}}document.getElementsByTagName('head')[0].appendChild(cpo);}());</script>Enable JavaScript and cookies to continue
From the response, it appears that the request is being blocked by Cloudflare with a "Just a moment..." challenge page. It seems like the request is being flagged as suspicious or not originating from a real browser.
In my understanding, LeetCode may have implemented some protection mechanisms, but I was wondering if there’s anything I can do on my end to resolve this, or if there’s a recommended way to authenticate and avoid getting blocked.
I really enjoy using DebugBench and would love to continue using it effectively with LeetCode. Any guidance or suggestions you can provide would be greatly appreciated!
Thanks a lot for your time and support.