You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 14:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Thu Dec 18, 2025 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27317991243,49cyclictest3904-21apt-get07:34:532
27076211119,60sleep027073-21munin-run07:09:390
25987210310,66sleep126016-21idleruntime07:05:031
273179910251,43cyclictest0-21swapper/209:45:042
273099910236,24cyclictest6247-21modprobe12:39:361
273099910236,24cyclictest6247-21modprobe12:39:361
273179910051,44cyclictest2362-21kworker/u18:0+events_unbound11:34:372
27317991003,68cyclictest22259-21apt-get10:09:562
27317991003,62cyclictest27806-21df07:10:312
27317991003,62cyclictest27806-21df07:10:312
2731799992,31cyclictest28971-21apt-get10:30:062
2731799982,59cyclictest452-21apt-get07:24:492
27309999641,46cyclictest0-21swapper/112:24:411
2730999962,43cyclictest20459-21cpu10:04:531
2731799953,81cyclictest14084-21apt-get08:04:522
2731799953,81cyclictest14084-21apt-get08:04:522
2731799953,61cyclictest1699-21apt-get12:24:552
2731799953,61cyclictest1699-21apt-get12:24:552
2730999933,62cyclictest28971-21apt-get10:29:571
27317999272,12cyclictest0-21swapper/209:34:392
27317999272,12cyclictest0-21swapper/209:34:392
27317999235,44cyclictest0-21swapper/208:04:412
27317999145,40cyclictest14397-21cstates11:24:562
27317999140,27cyclictest25763-21fschecks_time10:19:582
2730999913,45cyclictest17365-21fschecks_count09:54:591
2730999913,31cyclictest27627-21cut12:04:581
2730999912,35cyclictest29846-21fschecks_count07:15:021
27317999011,67cyclictest8994-21apt-get07:49:512
27317999011,67cyclictest8994-21apt-get07:49:512
2730999904,37cyclictest21719-21unixbench_singl11:45:141
2730999903,40cyclictest1194-21apt-get12:24:521
2730999903,40cyclictest1194-21apt-get12:24:521
2730999903,38cyclictest5106-21grep09:15:141
2730999903,36cyclictest651-21cut07:25:001
2730999903,36cyclictest32200-21/usr/sbin/munin10:40:021
2730999903,35cyclictest15067-21head08:05:331
2730999903,35cyclictest15067-21head08:05:331
27299999047,33cyclictest24775-21grep10:15:120
27317998945,37cyclictest0-21swapper/208:10:142
27317998941,41cyclictest0-21swapper/211:49:372
27317998917,31cyclictest26934-21latency_hist08:44:402
27309998938,9cyclictest0-21swapper/112:32:541
27309998938,9cyclictest0-21swapper/112:32:541
2730999893,40cyclictest19319-21kernelversion08:20:011
2730999893,39cyclictest12514-21fschecks_time07:59:591
2730999893,37cyclictest10496-21latency_hist07:54:411
2730999893,37cyclictest10496-21latency_hist07:54:411
2730999893,34cyclictest4252-21sort07:35:351
27309998928,16cyclictest0-21swapper/107:44:401
27309998928,14cyclictest0-21swapper/109:34:401
27309998928,14cyclictest0-21swapper/109:34:401
2730999892,34cyclictest31873-21proc_pri07:20:091
2730999892,30cyclictest25325-21latency_hist10:19:391
27317998842,37cyclictest0-21swapper/212:20:152
2731799883,39cyclictest19918-21head08:20:122
27309998837,43cyclictest0-21swapper/111:10:051
2730999883,35cyclictest350-21runrttasks12:17:011
2730999883,33cyclictest23791-21wc08:34:561
2730999883,33cyclictest20708-21latency_hist11:44:411
2730999883,32cyclictest3394-21cat10:50:031
2730999883,32cyclictest16794-21grep11:30:141
2730999883,30cyclictest14354-21latency09:45:041
2730999882,42cyclictest13118-21timerandwakeup09:40:121
2730999882,42cyclictest13118-21timerandwakeup09:40:121
2730999882,37cyclictest6013-21latency_hist10:59:391
2730999882,37cyclictest6013-21latency_hist10:59:391
2730999882,34cyclictest2591-21cat09:09:411
2730999882,31cyclictest962-21grep09:04:411
27309998822,14cyclictest0-21swapper/107:34:401
27299998845,33cyclictest381-21grep10:40:100
27317998742,38cyclictest0-21swapper/207:34:402
2731799873,35cyclictest350-21runrttasks08:33:212
2730999873,33cyclictest28668-21cmp10:29:401
2730999873,33cyclictest28668-21cmp10:29:401
2730999873,32cyclictest19004-21fschecks_count09:59:591
2730999873,31cyclictest22419-21fschecks_time08:29:591
27309998729,9cyclictest0-21swapper/111:55:111
27309998729,9cyclictest0-21swapper/111:55:111
2730999872,36cyclictest18640-21cron08:19:371
2731799864,39cyclictest12064-21munin-run09:39:372
2731799864,39cyclictest12064-21munin-run09:39:372
27317998643,36cyclictest0-21swapper/210:35:112
2731799863,41cyclictest20701-21idleruntime10:04:592
2731799863,38cyclictest12925-21ps09:40:092
2731799863,38cyclictest12925-21ps09:40:092
2731799863,33cyclictest1786-21taskset09:05:122
27317998621,58cyclictest2198-21wc10:45:122
27309998662,18cyclictest8487-21latency_hist07:49:411
2730999864,32cyclictest29704-21mailstats12:10:121
27309998635,7cyclictest0-21swapper/111:25:121
2730999863,37cyclictest2817-21latency_hist10:49:401
2730999863,35cyclictest31281-21switchtime10:35:111
2730999863,34cyclictest7627-21latency_hist11:04:391
2730999863,34cyclictest17532-21cpu11:34:541
2730999863,34cyclictest12123-21latency_hist07:59:401
2730999863,34cyclictest12123-21latency_hist07:59:401
2730999863,32cyclictest17123-21grep08:14:401
2730999863,31cyclictest30892-21mailstats08:55:101
2730999863,30cyclictest3986-21ls09:14:371
2730999862,35cyclictest28868-21grep08:50:001
2730999862,35cyclictest28868-21grep08:50:001
2730999862,35cyclictest26771-21unixbench_singl12:00:141
2730999862,35cyclictest16314-21xargs09:50:111
2730999862,32cyclictest28271-21cut07:10:451
2730999862,32cyclictest28271-21cut07:10:451
27299998665,16cyclictest1890-21ntp_offset10:45:050
2731799854,38cyclictest25324-21latency_hist10:19:392
2731799854,37cyclictest11700-21sendmail_mailtr11:15:112
2731799853,38cyclictest28377-21tail12:05:132
2731799853,24cyclictest205-21systemd-journal10:02:472
27317998517,26cyclictest10310-21cat11:10:142
2730999854,32cyclictest23759-21latency_hist10:14:411
2730999853,41cyclictest6785-21grep09:20:141
2730999853,37cyclictest23990-21latency_hist11:54:391
2730999853,33cyclictest12552-21latency_hist11:19:401
2730999852,37cyclictest8293-21needreboot11:05:061
2731799845,35cyclictest12789-21cstates11:19:552
2731799844,36cyclictest24907-21switchtime11:55:102
2731799844,36cyclictest24907-21switchtime11:55:102
2731799843,35cyclictest29608-21fschecks_time07:14:502
2731799843,29cyclictest27209-21grep08:44:582
2731799843,29cyclictest27209-21grep08:44:582
2731799843,29cyclictest19007-21latency_hist11:39:402
2731799843,28cyclictest7728-21latency_hist11:04:402
2731799843,24cyclictest5456-21/usr/sbin/munin12:35:062
2731799843,24cyclictest5456-21/usr/sbin/munin12:35:062
2731799843,23cyclictest24654-21/usr/sbin/munin08:35:152
2730999843,31cyclictest20926-21kernelversion08:25:051
2730999842,35cyclictest26399-21grep10:20:121
2730999842,30cyclictest12793-21cstates11:19:541
2730999842,29cyclictest25812-21latency08:40:041
27309998420,25cyclictest27849-21timerwakeupswit08:45:121
27309998420,25cyclictest27849-21timerwakeupswit08:45:121
2731799834,38cyclictest8736-21apt-key07:49:402
2731799833,35cyclictest29468-21timerwakeupswit08:50:132
2731799833,35cyclictest29468-21timerwakeupswit08:50:132
27317998332,44cyclictest0-21swapper/209:50:052
27317998332,43cyclictest0-21swapper/208:54:542
27299998357,18cyclictest924-21cron12:29:370
27299998357,18cyclictest924-21cron12:29:370
2731799823,38cyclictest3373-21ls12:29:522
2731799823,38cyclictest3373-21ls12:29:522
2731799823,32cyclictest7045-21wc07:44:412
2731799823,24cyclictest31774-21/usr/sbin/munin07:20:052
2731799823,24cyclictest28459-21modprobe10:29:372
2731799823,24cyclictest28459-21modprobe10:29:372
2730999823,38cyclictest350-21runrttasks09:24:551
2730999823,38cyclictest350-21runrttasks09:24:551
27317998158,15cyclictest11931-21cron07:59:372
27317998158,15cyclictest11931-21cron07:59:372
27317998136,38cyclictest0-21swapper/210:50:112
2731799813,30cyclictest8082-21ps09:25:102
2731799813,30cyclictest8082-21ps09:25:102
2731799813,25cyclictest1120-21idleruntime-cro10:44:382
2731799813,24cyclictest4878-21/usr/sbin/munin10:54:592
2731799813,24cyclictest4878-21/usr/sbin/munin10:54:592
2731799812,24cyclictest20895-21/usr/sbin/munin08:25:042
2731799812,23cyclictest17510-21/usr/sbin/munin09:55:012
27309998135,8cyclictest0-21swapper/109:35:251
27309998135,8cyclictest0-21swapper/109:35:251
2617428126,35sleep30-21swapper/307:05:103
27317998044,29cyclictest0-21swapper/209:10:022
2731799804,28cyclictest350-21runrttasks11:06:102
2731799803,32cyclictest32111-21fschecks_time08:59:592
2731799803,28cyclictest19828-21/usr/sbin/munin11:40:132
2731799803,26cyclictest6458-21/usr/sbin/munin09:20:092
2731799803,26cyclictest30548-21date12:14:402
2731799803,25cyclictest22980-21cat11:50:042
2731799802,27cyclictest17639-21/usr/sbin/munin08:15:012
27299998060,14cyclictest30003-21unixbench_singl12:10:150
27299998059,15cyclictest10567-21latency_hist09:34:400
27299998059,15cyclictest10567-21latency_hist09:34:400
27317997961,14cyclictest27280-21cyclictest12:15:002
2731799794,33cyclictest25815-21/usr/sbin/munin11:59:542
27299997959,14cyclictest350-21runrttasks08:44:590
27299997959,14cyclictest350-21runrttasks08:44:590
27299997948,9cyclictest181rcu_preempt11:19:370
27299997935,18cyclictest181rcu_preempt09:15:100
2733599783,62cyclictest9013-21chrt07:49:593
2733599783,62cyclictest9013-21chrt07:49:593
2731799782,28cyclictest4343-21cstates09:14:552
27299997857,15cyclictest17444-21latency_hist11:34:410
27299997857,14cyclictest14189-21latency_hist11:24:400
27299997835,24cyclictest181rcu_preempt08:34:400
2729999783,44cyclictest15-21ksoftirqd/012:21:260
27299997759,11cyclictest8161-21runrttasks11:05:030
27299997758,13cyclictest1-21systemd10:22:470
27299997743,17cyclictest181rcu_preempt12:38:330
27299997743,17cyclictest181rcu_preempt12:38:330
27299997741,25cyclictest19556-21xargs10:00:120
27299997740,10cyclictest181rcu_preempt07:35:460
27299997733,12cyclictest181rcu_preempt11:55:040
27299997733,12cyclictest181rcu_preempt11:55:040
2729999772,41cyclictest15-21ksoftirqd/010:55:080
2729999772,41cyclictest15-21ksoftirqd/010:55:080
27299997654,14cyclictest21757-21latency_hist10:09:410
27299997646,10cyclictest181rcu_preempt07:59:400
27299997646,10cyclictest181rcu_preempt07:59:400
27299997644,7cyclictest0-21swapper/009:11:550
27299997629,10cyclictest181rcu_preempt07:12:120
27299997629,10cyclictest181rcu_preempt07:12:120
27299997555,12cyclictest181rcu_preempt10:30:300
2729999743,35cyclictest15127-21sort11:25:130
2729999743,29cyclictest8488-21apt-get07:49:500
2729999743,29cyclictest8488-21apt-get07:49:500
27299997429,24cyclictest181rcu_preempt09:49:370
27299997341,8cyclictest181rcu_preempt11:46:410
27299997339,12cyclictest181rcu_preempt09:34:550
27299997339,12cyclictest181rcu_preempt09:34:550
27299997335,11cyclictest181rcu_preempt11:12:130
27299997330,14cyclictest181rcu_preempt08:24:560
27299997328,12cyclictest181rcu_preempt08:34:570
27299997326,24cyclictest28067-21find12:05:090
2733599722,35cyclictest13310-21chrt09:40:133
2733599722,35cyclictest13310-21chrt09:40:133
2731799723,27cyclictest27280-21cyclictest07:09:432
27299997248,17cyclictest0-21swapper/008:44:390
27299997239,12cyclictest181rcu_preempt07:20:170
2729999723,55cyclictest747-21modprobe09:04:370
2729999723,31cyclictest13575-21modprobe08:04:370
27299997228,13cyclictest181rcu_preempt07:45:050
27299997156,11cyclictest27280-21cyclictest12:01:340
27299997151,14cyclictest3516-21apt-key07:34:400
27299997139,15cyclictest181rcu_preempt08:09:490
2729999713,50cyclictest22818-21fschecks_time11:50:000
2729999713,33cyclictest350-21runrttasks11:39:230
27299997053,13cyclictest211rcuc/010:24:500
27299997053,13cyclictest211rcuc/010:24:500
27299997044,9cyclictest181rcu_preempt11:04:400
27299997034,9cyclictest181rcu_preempt09:50:130
2733599693,28cyclictest25410-21chrt08:39:523
2733599692,29cyclictest7860-21chrt09:25:033
2733599692,29cyclictest7860-21chrt09:25:033
27299996956,9cyclictest58-21kcompactd007:28:570
27299996947,9cyclictest181rcu_preempt08:15:050
27299996944,8cyclictest181rcu_preempt08:09:400
27299996944,8cyclictest181rcu_preempt08:09:400
27299996942,9cyclictest181rcu_preempt10:34:550
27299996939,12cyclictest181rcu_preempt08:54:400
27299996939,12cyclictest181rcu_preempt08:54:400
2729999693,33cyclictest19244-21cstates11:39:560
27299996933,11cyclictest181rcu_preempt09:19:590
27299996920,22cyclictest29914-21perl07:15:030
2729999685,43cyclictest30650-21/usr/sbin/munin12:15:020
2729999685,29cyclictest15-21ksoftirqd/008:54:480
2729999684,45cyclictest3482-21fschecks_count12:29:570
2729999684,45cyclictest3482-21fschecks_count12:29:570
27299996842,11cyclictest181rcu_preempt07:40:120
2729999683,42cyclictest15-21ksoftirqd/009:05:130
2729999683,38cyclictest3887-21unixbench_multi10:50:130
27299996831,17cyclictest181rcu_preempt10:10:260
2733599674,24cyclictest12892-21taskset11:19:593
27299996737,12cyclictest181rcu_preempt09:55:000
2733599664,23cyclictest24220-21chrt08:35:063
2733599662,25cyclictest31642-21chrt07:20:023
2733599662,22cyclictest4869-21chrt09:15:113
27299996646,14cyclictest0-21swapper/008:24:400
27299996644,8cyclictest181rcu_preempt09:40:030
27299996644,8cyclictest181rcu_preempt09:40:030
2733599643,23cyclictest2481-21taskset09:09:403
2729999645,44cyclictest8257-21threads09:25:120
2729999645,44cyclictest8257-21threads09:25:120
2733599633,22cyclictest19156-21chrt11:39:553
2733599632,23cyclictest15717-21taskset09:49:573
2733599632,22cyclictest26370-21chrt12:00:083
2733599623,24cyclictest7310-21taskset11:01:243
2733599623,24cyclictest27982-21chrt12:05:063
2733599623,24cyclictest22702-21chrt08:30:083
2733599623,21cyclictest24426-21chrt10:15:083
2733599622,25cyclictest10268-21taskset09:32:283
2733599622,25cyclictest10268-21taskset09:32:283
2733599622,23cyclictest31996-21chrt12:19:353
2733599622,22cyclictest21933-21chrt08:29:393
2733599622,21cyclictest30211-21chrt07:15:123
2733599613,26cyclictest8252-21taskset07:46:433
2733599613,25cyclictest909-21chrt10:42:173
2733599613,23cyclictest31635-21chrt10:36:053
2733599613,22cyclictest20971-21chrt10:05:073
2733599613,22cyclictest12164-21taskset11:16:473
2733599613,21cyclictest13407-21chrt08:02:303
2733599612,25cyclictest3320-21chrt12:29:523
2733599612,25cyclictest3320-21chrt12:29:523
2733599612,25cyclictest2257-21taskset10:45:123
2733599612,24cyclictest3862-21chrt09:13:123
2733599612,24cyclictest27174-21taskset08:44:583
2733599612,24cyclictest27174-21taskset08:44:583
2733599612,22cyclictest2971-21chrt12:28:513
2733599612,22cyclictest2971-21chrt12:28:513
2733599612,22cyclictest23124-21chrt11:50:083
2733599612,22cyclictest17667-21chrt11:34:573
2733599612,22cyclictest11769-21chrt07:57:323
2733599612,22cyclictest11769-21chrt07:57:323
2733599612,21cyclictest28347-21taskset07:10:473
2733599612,21cyclictest28347-21taskset07:10:473
2733599612,21cyclictest17150-21taskset11:33:353
27335996120,9cyclictest140902chrt08:04:493
27335996120,9cyclictest140902chrt08:04:493
27335996110,40cyclictest0-21swapper/311:47:073
2733599603,23cyclictest5037-21chrt07:37:033
2733599603,23cyclictest1676-21chrt07:28:163
2733599603,22cyclictest30114-21chrt12:11:053
2733599603,22cyclictest16843-21chrt08:12:143
2733599603,21cyclictest5582-21chrt10:55:213
2733599603,21cyclictest5582-21chrt10:55:213
2733599603,21cyclictest28282-21chrt10:27:233
2733599603,21cyclictest28282-21chrt10:27:233
2733599603,20cyclictest29623-21chrt08:50:203
2733599603,20cyclictest29623-21chrt08:50:203
2733599603,20cyclictest15128-21chrt11:25:133
2733599603,15cyclictest32673-21chrt12:20:043
2733599602,25cyclictest19827-21chrt10:00:383
2733599602,24cyclictest6065-21chrt12:37:173
2733599602,24cyclictest6065-21chrt12:37:173
2733599602,23cyclictest6559-21chrt07:40:353
2733599602,23cyclictest3318-21chrt07:33:473
2733599602,23cyclictest32407-21taskset09:00:083
2733599602,23cyclictest22268-21chrt10:09:533
2733599602,22cyclictest6973-21chrt09:21:223
2733599602,22cyclictest19950-21chrt08:20:243
2733599602,21cyclictest9130-21chrt11:09:143
2733599602,21cyclictest25409-21chrt11:58:053
2733599602,21cyclictest25409-21chrt11:58:053
2733599602,21cyclictest15261-21chrt09:49:243
2733599602,21cyclictest10598-21chrt11:12:263
2733599602,20cyclictest31475-21chrt08:58:343
2733599592,23cyclictest30112-21taskset10:32:083
2733599592,22cyclictest4033-21chrt10:51:043
2733599592,21cyclictest26606-21chrt10:21:363
2733599592,21cyclictest18253-21chrt09:56:293
2733599592,21cyclictest11831-21chrt09:36:533
2733599592,21cyclictest11831-21chrt09:36:533
2733599592,20cyclictest18370-21chrt08:16:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional