Failed Jobs

Enqueued 0 / 0 Scheduled 0 Processing 0 Succeeded 3,320,351 Failed 9,618 Deleted 95 Awaiting 0

Failed jobs do not become expired to allow you to re-queue them without any time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete) attribute to delete them automatically.

Id	Job	Failed
#1884188	DeleteDraftJobsTask.StartDeleteDraftJobs An exception occurred during performance of the job. Fewer details...	05/16/2025 03:25:15
#1884188	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at MR.Schedules.DeleteDraftJobsTask.StartDeleteDraftJobs() in /home/azureuser/sn1/src/BackEnd/MR.Schedules/Tasks/DeleteDraftJobsTask.cs:line 38 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr args, BindingFlags invokeAttr)`
#1884187	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	05/16/2025 03:25:14
#1884187	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at MR.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/sn1/src/BackEnd/MR.Schedules/Tasks/SendNotificationsTask.cs:line 37 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr args, BindingFlags invokeAttr)`
#1878187	DeleteDraftJobsTask.StartDeleteDraftJobs An exception occurred during performance of the job. More details...	05/14/2025 18:09:13
#1878187	System.InvalidOperationException Could not get a value of the job parameter `CurrentUICulture`. See inner exception for details. System.InvalidOperationException: Could not get a value of the job parameter `CurrentUICulture`. See inner exception for details. ---> Npgsql.PostgresException (0x80004005): 53300: sorry, too many clients already at Npgsql.Internal.NpgsqlConnector.<ReadMessage>g__ReadMessageLong\|213_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage) at Npgsql.Internal.NpgsqlConnector.Authenticate(String username, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.<Open>g__OpenCore\|193_1(NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.ConnectorPool.OpenNewConnector(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.ConnectorPool.<Get>g__RentAsync\|31_0(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.<Open>g__OpenAsync\|45_0(Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.Open() at Hangfire.PostgreSql.PostgreSqlStorage.CreateAndOpenConnection() at Hangfire.PostgreSql.PostgreSqlStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func) at Hangfire.PostgreSql.PostgreSqlConnection.GetJobParameter(String id, String name) at Hangfire.Server.PerformContext.GetJobParameter[T](String name, Boolean allowStale) Exception data: Severity: FATAL SqlState: 53300 MessageText: sorry, too many clients already File: proc.c Line: 360 Routine: InitProcess --- End of inner exception stack trace --- at Hangfire.Server.PerformContext.GetJobParameter[T](String name, Boolean allowStale) at Hangfire.CaptureCultureAttribute.OnPerforming(PerformingContext context) at Hangfire.Profiling.ProfilerExtensions.InvokeAction[TInstance](InstanceAction`1 tuple) at Hangfire.Profiling.SlowLogProfiler.InvokeMeasured[TInstance,TResult](TInstance instance, Func`2 action, String message) at Hangfire.Profiling.ProfilerExtensions.InvokeMeasured[TInstance](IProfiler profiler, TInstance instance, Action`1 action, String message) at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation)
#1875943	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	05/14/2025 05:50:51
#1875943	System.Threading.Tasks.TaskCanceledException The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. ---> System.TimeoutException: The operation was canceled. ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException (125): Operation canceled --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token) at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken) at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token) at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken) at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token) at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) --- End of inner exception stack trace --- --- End of inner exception stack trace --- at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at MR.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/sn1/src/BackEnd/MR.Schedules/Tasks/SendNotificationsTask.cs:line 44 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr args, BindingFlags invokeAttr)
#1873117	ExecuteBackgroundJobTask.StartExecuteBackgroundJob An exception occurred during performance of the job. More details...	05/13/2025 14:15:05
#1873117	System.InvalidOperationException Could not get a value of the job parameter `CurrentCulture`. See inner exception for details. System.InvalidOperationException: Could not get a value of the job parameter `CurrentCulture`. See inner exception for details. ---> Npgsql.PostgresException (0x80004005): 53300: sorry, too many clients already at Npgsql.Internal.NpgsqlConnector.<ReadMessage>g__ReadMessageLong\|213_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage) at Npgsql.Internal.NpgsqlConnector.Authenticate(String username, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.<Open>g__OpenCore\|193_1(NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.ConnectorPool.OpenNewConnector(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.ConnectorPool.<Get>g__RentAsync\|31_0(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.<Open>g__OpenAsync\|45_0(Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.Open() at Hangfire.PostgreSql.PostgreSqlStorage.CreateAndOpenConnection() at Hangfire.PostgreSql.PostgreSqlStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func) at Hangfire.PostgreSql.PostgreSqlConnection.GetJobParameter(String id, String name) at Hangfire.Server.PerformContext.GetJobParameter[T](String name, Boolean allowStale) Exception data: Severity: FATAL SqlState: 53300 MessageText: sorry, too many clients already File: proc.c Line: 360 Routine: InitProcess --- End of inner exception stack trace --- at Hangfire.Server.PerformContext.GetJobParameter[T](String name, Boolean allowStale) at Hangfire.CaptureCultureAttribute.OnPerforming(PerformingContext context) at Hangfire.Profiling.ProfilerExtensions.InvokeAction[TInstance](InstanceAction`1 tuple) at Hangfire.Profiling.SlowLogProfiler.InvokeMeasured[TInstance,TResult](TInstance instance, Func`2 action, String message) at Hangfire.Profiling.ProfilerExtensions.InvokeMeasured[TInstance](IProfiler profiler, TInstance instance, Action`1 action, String message) at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation)
#1873116	DeleteDraftJobsTask.StartDeleteDraftJobs An exception occurred during performance of the job. More details...	05/13/2025 14:15:05
#1873116	System.InvalidOperationException Could not get a value of the job parameter `CurrentCulture`. See inner exception for details. System.InvalidOperationException: Could not get a value of the job parameter `CurrentCulture`. See inner exception for details. ---> Npgsql.PostgresException (0x80004005): 53300: sorry, too many clients already at Npgsql.Internal.NpgsqlConnector.<ReadMessage>g__ReadMessageLong\|213_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage) at Npgsql.Internal.NpgsqlConnector.Authenticate(String username, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.<Open>g__OpenCore\|193_1(NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.ConnectorPool.OpenNewConnector(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.ConnectorPool.<Get>g__RentAsync\|31_0(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.<Open>g__OpenAsync\|45_0(Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.Open() at Hangfire.PostgreSql.PostgreSqlStorage.CreateAndOpenConnection() at Hangfire.PostgreSql.PostgreSqlStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func) at Hangfire.PostgreSql.PostgreSqlConnection.GetJobParameter(String id, String name) at Hangfire.Server.PerformContext.GetJobParameter[T](String name, Boolean allowStale) Exception data: Severity: FATAL SqlState: 53300 MessageText: sorry, too many clients already File: proc.c Line: 360 Routine: InitProcess --- End of inner exception stack trace --- at Hangfire.Server.PerformContext.GetJobParameter[T](String name, Boolean allowStale) at Hangfire.CaptureCultureAttribute.OnPerforming(PerformingContext context) at Hangfire.Profiling.ProfilerExtensions.InvokeAction[TInstance](InstanceAction`1 tuple) at Hangfire.Profiling.SlowLogProfiler.InvokeMeasured[TInstance,TResult](TInstance instance, Func`2 action, String message) at Hangfire.Profiling.ProfilerExtensions.InvokeMeasured[TInstance](IProfiler profiler, TInstance instance, Action`1 action, String message) at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation)
#1870314	DeleteDraftJobsTask.StartDeleteDraftJobs An exception occurred during performance of the job. More details...	05/12/2025 20:48:12
#1870314	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at MR.Schedules.DeleteDraftJobsTask.StartDeleteDraftJobs() in /var/www/ManageRight/src/BackEnd/MR.Schedules/Tasks/DeleteDraftJobsTask.cs:line 45 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr args, BindingFlags invokeAttr)`
#1870313	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	05/12/2025 20:48:12
#1870313	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at MR.Schedules.SendNotificationsTask.StartJobSendNotifications() in /var/www/ManageRight/src/BackEnd/MR.Schedules/Tasks/SendNotificationsTask.cs:line 44 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr args, BindingFlags invokeAttr)`
#1870265	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	05/12/2025 20:32:43
#1870265	System.Net.Http.HttpRequestException Response status code does not indicate success: 422 (Unprocessable Entity). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 422 (Unprocessable Entity). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at MR.Schedules.SendNotificationsTask.StartJobSendNotifications() in /var/www/ManageRight/src/BackEnd/MR.Schedules/Tasks/SendNotificationsTask.cs:line 44 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr args, BindingFlags invokeAttr)`
#1870235	DeleteDraftJobsTask.StartDeleteDraftJobs An exception occurred during performance of the job. More details...	05/12/2025 20:22:31
#1870235	System.Net.Http.HttpRequestException Response status code does not indicate success: 500 (Internal Server Error). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 500 (Internal Server Error). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at MR.Schedules.DeleteDraftJobsTask.StartDeleteDraftJobs() in /var/www/ManageRight/src/BackEnd/MR.Schedules/Tasks/DeleteDraftJobsTask.cs:line 45 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr) at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr args, BindingFlags invokeAttr)`

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException

System.InvalidOperationException

System.Threading.Tasks.TaskCanceledException

System.InvalidOperationException

System.InvalidOperationException

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException