INFO:__main__:Worker started. INFO:__main__:Parameters: Namespace(history_uuid='ec515b48-e43f-11ee-83e1-fe3e40614918') INFO:__main__:conversion_history INFO:__main__: INFO:__main__:current_user from db Quang Linh Đỗ Hoàng ERROR:__main__:processing tts_history: ec515b48-e43f-11ee-83e1-fe3e40614918 ERROR:__main__:current user_id: 511 INFO:__main__:Try to download and decrypt if needed for ec515b48-e43f-11ee-83e1-fe3e40614918 INFO:__main__:File downloaded successfully to: tmp/contabo_uploaded/511/20240317_092245_362388/kiem lai 3.docx INFO:Job: convert_and_send_result ec515b48-e43f-11ee-83e1-fe3e40614918:Convert TTS and send result START... INFO:Job: convert_and_send_result ec515b48-e43f-11ee-83e1-fe3e40614918:conversion_history_uuid ec515b48-e43f-11ee-83e1-fe3e40614918 INFO:Job: convert_and_send_result ec515b48-e43f-11ee-83e1-fe3e40614918:input_file_path tmp/contabo_uploaded/511/20240317_092245_362388/kiem lai 3.docx INFO:Job: convert_and_send_result ec515b48-e43f-11ee-83e1-fe3e40614918:model_price_rate 2 INFO:Job: convert_and_send_result ec515b48-e43f-11ee-83e1-fe3e40614918:file_extension docx INFO:Job: convert_and_send_result ec515b48-e43f-11ee-83e1-fe3e40614918:current estimated_total_credits to calculate lock: 43812 INFO:Job: convert_and_send_result ec515b48-e43f-11ee-83e1-fe3e40614918:current_estimate_credits: 43812 0-kiem lai 3 tmp/contabo_uploaded/511/20240317_092245_362388/0-kiem lai 3.mp3 1-kiem lai 3 tmp/contabo_uploaded/511/20240317_092245_362388/1-kiem lai 3.mp3 INFO:app.tts_common.services.speech:Parent process: Process-2 START INFO:app.tts_common.services.speech:setup_api_key of model: gpt-3.5 INFO:app.tts_common.services.speech:Parent process: Process-3 START INFO:app.tts_common.services.speech:setup_api_key of model: gpt-3.5 INFO:app.tts_common.services.speech:count_all_active_key: 1260 INFO:app.tts_common.services.speech:Requesting openAI with key: ********************3tZFG2Zo9W4hjfK5Nm2EsUv2X INFO:app.tts_common.services.speech:count_all_active_key: 1260 INFO:app.tts_common.services.speech:Requesting openAI with key: ********************kjinmzdn8u2dHBPF5M4Y89wjd INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.services.speech:Requesting openAI with key: ********************3tZFG2Zo9W4hjfK5Nm2EsUv2X INFO:app.tts_common.services.speech:Requesting openAI with key: ********************kjinmzdn8u2dHBPF5M4Y89wjd INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.services.speech:Requesting openAI with key: ********************3tZFG2Zo9W4hjfK5Nm2EsUv2X INFO:app.tts_common.services.speech:Requesting openAI with key: ********************kjinmzdn8u2dHBPF5M4Y89wjd INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.services.speech:Requesting openAI with key: ********************3tZFG2Zo9W4hjfK5Nm2EsUv2X INFO:app.tts_common.services.speech:Requesting openAI with key: ********************kjinmzdn8u2dHBPF5M4Y89wjd INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 429 Too Many Requests" ERROR:app.tts_common.services.speech:A 429 status code was received; we should back off a bit. INFO:app.tts_common.crud.crud_openai_api_key:set_usable_at: sess-Xh11k6a2npNuwVJ3tZFG2Zo9W4hjfK5Nm2EsUv2X - gpt-3.5 - 2024-03-17 09:25:15.736456 INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 429 Too Many Requests" ERROR:app.tts_common.services.speech:A 429 status code was received; we should back off a bit. INFO:app.tts_common.crud.crud_openai_api_key:set_usable_at: sess-iGef6UotVblspIvkjinmzdn8u2dHBPF5M4Y89wjd - gpt-3.5 - 2024-03-17 09:25:15.853437 INFO:app.tts_common.services.speech:Requesting openAI with key: ********************kxSDIAsEqWiuLY565AbP6hBOl INFO:app.tts_common.services.speech:Requesting openAI with key: ********************FvVLEMI339uiJvPKKb7HTQJmc INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.services.speech:Requesting openAI with key: ********************kxSDIAsEqWiuLY565AbP6hBOl INFO:app.tts_common.services.speech:Requesting openAI with key: ********************FvVLEMI339uiJvPKKb7HTQJmc INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.services.speech:Requesting openAI with key: ********************kxSDIAsEqWiuLY565AbP6hBOl INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.services.speech:Requesting openAI with key: ********************FvVLEMI339uiJvPKKb7HTQJmc INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.services.speech:Requesting openAI with key: ********************kxSDIAsEqWiuLY565AbP6hBOl INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 429 Too Many Requests" ERROR:app.tts_common.services.speech:A 429 status code was received; we should back off a bit. INFO:app.tts_common.crud.crud_openai_api_key:set_usable_at: sess-X0GlFyvfovJXCDOkxSDIAsEqWiuLY565AbP6hBOl - gpt-3.5 - 2024-03-17 09:25:28.563004 INFO:app.tts_common.services.speech:Requesting openAI with key: ********************FvVLEMI339uiJvPKKb7HTQJmc INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 429 Too Many Requests" ERROR:app.tts_common.services.speech:A 429 status code was received; we should back off a bit. INFO:app.tts_common.crud.crud_openai_api_key:set_usable_at: sess-DtSbpidkBMSaQCCFvVLEMI339uiJvPKKb7HTQJmc - gpt-3.5 - 2024-03-17 09:25:28.804010 INFO:app.tts_common.services.speech:Requesting openAI with key: ********************6dey6Be5ceY7U1j6GQLaxcCUt INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.services.speech:Requesting openAI with key: ********************U34fEU7VstOTAU13FLetEiakj INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 400 Bad Request" ERROR:app.tts_common.services.speech:Another non-200-range status code was received ERROR:app.tts_common.services.speech:400 ERROR:app.tts_common.services.speech: INFO:app.tts_common.helpers.translator_helper:get_boss_key response INFO:app.tts_common.helpers.translator_helper: INFO:app.tts_common.helpers.translator_helper:https://iv0fyyh85j.execute-api.eu-central-1.amazonaws.com/prod/oakey/boss/f6e5740d-099c-4084-a454-bb339b407128 INFO:app.tts_common.helpers.translator_helper:{'success': True, 'api_key': 'sk-xzXro7PxD2MpQzw44b6329C566F24e129cBeBb05A99fDb1c', 'partitionKey': 'bossKey', 'created_at': '2023-08-01 11:02:34', 'id': 2, 'base_url': 'https://aihubmix.com/v1'} INFO:app.tts_common.services.speech:Using boss key in request_text_openai_by_piece INFO:app.tts_common.services.speech:Requesting openAI with key: ********************b6329C566F24e129cBeBb05A99fDb1c INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 401 Unauthorized" ERROR:app.tts_common.services.speech:The API key is invalid ERROR:app.tts_common.services.speech:Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-xzXro***************************************Db1c. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}} INFO:app.tts_common.crud.crud_openai_api_key:deactivate_api_key: sk-xzXro7PxD2MpQzw44b6329C566F24e129cBeBb05A99fDb1c - gpt-3.5 INFO:app.tts_common.helpers.translator_helper:get_boss_key response INFO:app.tts_common.helpers.translator_helper: INFO:app.tts_common.helpers.translator_helper:https://iv0fyyh85j.execute-api.eu-central-1.amazonaws.com/prod/oakey/boss/f6e5740d-099c-4084-a454-bb339b407128 INFO:app.tts_common.helpers.translator_helper:{'success': True, 'api_key': 'sk-xzXro7PxD2MpQzw44b6329C566F24e129cBeBb05A99fDb1c', 'partitionKey': 'bossKey', 'created_at': '2023-08-01 11:02:34', 'id': 2, 'base_url': 'https://aihubmix.com/v1'} INFO:app.tts_common.services.speech:Using boss key in request_text_openai_by_piece INFO:app.tts_common.services.speech:Requesting openAI with key: ********************b6329C566F24e129cBeBb05A99fDb1c INFO:httpx:HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 401 Unauthorized" ERROR:app.tts_common.services.speech:The API key is invalid ERROR:app.tts_common.services.speech:Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-xzXro***************************************Db1c. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}} INFO:app.tts_common.crud.crud_openai_api_key:deactivate_api_key: sk-xzXro7PxD2MpQzw44b6329C566F24e129cBeBb05A99fDb1c - gpt-3.5 INFO:app.tts_common.helpers.tts_history_helper:add_percent_complete: ec515b48-e43f-11ee-83e1-fe3e40614918 - 50% INFO:[CRUD History]:add_percent_complete: ec515b48-e43f-11ee-83e1-fe3e40614918 - 50% INFO:app.tts_common.services.speech:Parent process: Process-2 END complete_count 0 total_pieces 2 INFO:app.tts_common.helpers.tts_history_helper:add_percent_complete: ec515b48-e43f-11ee-83e1-fe3e40614918 - 100% INFO:[CRUD History]:add_percent_complete: ec515b48-e43f-11ee-83e1-fe3e40614918 - 100% INFO:app.tts_common.services.speech:Parent process: Process-3 END complete_count 1 total_pieces 2 ERROR:__main__:Convert and send result failed ERROR:__main__:[Errno 2] No such file or directory: 'tmp/contabo_uploaded/511/20240317_092245_362388/0-kiem lai 3.mp3' Traceback (most recent call last): File "/app/worker.py", line 153, in main convert_result = await convert_and_send_result( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/app/tts_common/jobs/convert_and_send_result.py", line 151, in convert_and_send_result audio = AudioSegment.from_mp3(file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/audio_segment.py", line 796, in from_mp3 return cls.from_file(file, 'mp3', parameters=parameters) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/audio_segment.py", line 651, in from_file file, close_file = _fd_or_path_or_tempfile(file, 'rb', tempfile=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/utils.py", line 60, in _fd_or_path_or_tempfile fd = open(fd, mode=mode) ^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'tmp/contabo_uploaded/511/20240317_092245_362388/0-kiem lai 3.mp3' ERROR:__main__:Traceback (most recent call last): File "/app/worker.py", line 153, in main convert_result = await convert_and_send_result( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/app/tts_common/jobs/convert_and_send_result.py", line 151, in convert_and_send_result audio = AudioSegment.from_mp3(file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/audio_segment.py", line 796, in from_mp3 return cls.from_file(file, 'mp3', parameters=parameters) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/audio_segment.py", line 651, in from_file file, close_file = _fd_or_path_or_tempfile(file, 'rb', tempfile=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/utils.py", line 60, in _fd_or_path_or_tempfile fd = open(fd, mode=mode) ^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'tmp/contabo_uploaded/511/20240317_092245_362388/0-kiem lai 3.mp3' Traceback (most recent call last): File "/app/worker.py", line 153, in main convert_result = await convert_and_send_result( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/app/tts_common/jobs/convert_and_send_result.py", line 151, in convert_and_send_result audio = AudioSegment.from_mp3(file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/audio_segment.py", line 796, in from_mp3 return cls.from_file(file, 'mp3', parameters=parameters) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/audio_segment.py", line 651, in from_file file, close_file = _fd_or_path_or_tempfile(file, 'rb', tempfile=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/utils.py", line 60, in _fd_or_path_or_tempfile fd = open(fd, mode=mode) ^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'tmp/contabo_uploaded/511/20240317_092245_362388/0-kiem lai 3.mp3' ERROR:__main__:ec515b48-e43f-11ee-83e1-fe3e40614918 Traceback (most recent call last): File "/app/worker.py", line 153, in main convert_result = await convert_and_send_result( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/app/tts_common/jobs/convert_and_send_result.py", line 151, in convert_and_send_result audio = AudioSegment.from_mp3(file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/audio_segment.py", line 796, in from_mp3 return cls.from_file(file, 'mp3', parameters=parameters) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/audio_segment.py", line 651, in from_file file, close_file = _fd_or_path_or_tempfile(file, 'rb', tempfile=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydub/utils.py", line 60, in _fd_or_path_or_tempfile fd = open(fd, mode=mode) ^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'tmp/contabo_uploaded/511/20240317_092245_362388/0-kiem lai 3.mp3' INFO:app.tts_common.helpers.email_helper:Send translated file failed notice email INFO:app.tts_common.helpers.email_helper:sent to message to kafka topic: tts-emails-prod INFO:Firebase client:Add notification to firebase START INFO:Firebase client:Add notification to firebase END INFO:__main__:Recalculate user locked credit INFO:__main__:Convert and send result END results save text pieces to files prepare to join audios Produced event to topic tts-emails-prod: value = {"function": "send_translated_file_failed_notice", "email_to": "dolinhluat@gmail.com", "recipient_name": "Quang Linh \u0110\u1ed7 Ho\u00e0ng", "file_name": "kiem lai 3.docx"}