You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 15:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Fri Jun 05, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13797218413,123sleep11-21systemd06:59:451
13896999978,14cyclictest352-21runrttasks12:25:042
13883999466,19cyclictest20790-21diskmemload09:11:530
13883999466,19cyclictest20790-21diskmemload09:11:530
13896999269,15cyclictest0-21swapper/208:19:442
13896999269,15cyclictest0-21swapper/208:19:442
13896999249,12cyclictest23759-21ssh09:57:542
13896999050,10cyclictest181rcu_preempt09:40:172
13883999054,11cyclictest181rcu_preempt12:08:060
13883999054,11cyclictest181rcu_preempt12:08:060
128782909,55sleep012402-21/usr/sbin/munin06:55:150
13891998970,13cyclictest17016-21sh11:35:011
13883998866,14cyclictest5261-21fschecks_time11:15:030
13896998665,14cyclictest7116-21idleruntime-cro10:24:422
13896998665,14cyclictest7116-21idleruntime-cro10:24:422
13896998651,13cyclictest6364-21ssh12:10:432
13896998642,12cyclictest181rcu_preempt07:49:542
13891998664,16cyclictest9779-21fschecks_count09:35:031
13891998664,16cyclictest9779-21fschecks_count09:35:031
13896998554,8cyclictest5721-21ssh09:28:162
13883998558,18cyclictest3442-21latency08:05:070
13883998558,18cyclictest3442-21latency08:05:070
1263928512,42sleep212689-21cut06:55:092
1389699846,65cyclictest13785-21kworker/u19:0+ext4-rsv-conversion08:45:062
13891998465,13cyclictest8268-21sh10:26:091
13883998463,14cyclictest13587-21sh11:29:430
13883998456,19cyclictest1119-21latency_hist10:14:450
1389199837,41cyclictest31-21ksoftirqd/111:19:351
13883998362,12cyclictest0-21swapper/011:06:090
13883998362,12cyclictest0-21swapper/011:06:090
13883998356,9cyclictest181rcu_preempt10:26:020
13883998350,12cyclictest181rcu_preempt07:44:450
13883998330,26cyclictest0-21swapper/009:14:580
1389199829,36cyclictest31-21ksoftirqd/107:23:391
13891998260,16cyclictest487-21sh11:06:201
13891998260,16cyclictest487-21sh11:06:201
13891998210,36cyclictest31-21ksoftirqd/111:04:401
13891998210,36cyclictest31-21ksoftirqd/111:04:401
13883998254,9cyclictest181rcu_preempt10:39:460
13883998247,12cyclictest181rcu_preempt11:50:030
13883998243,19cyclictest181rcu_preempt11:48:370
630828110,61sleep06313-21ssh09:29:440
13896998166,8cyclictest0-21swapper/211:24:532
13896998159,12cyclictest0-21swapper/211:10:012
13896998153,19cyclictest21723-21threads07:20:162
13896998147,13cyclictest21448-21ssh10:48:022
13883998151,11cyclictest181rcu_preempt09:35:030
13883998151,11cyclictest181rcu_preempt09:35:030
13883998120,40cyclictest0-21swapper/008:10:040
13883998120,40cyclictest0-21swapper/008:10:040
13896998034,34cyclictest19098-21apt07:14:422
13896998034,34cyclictest19098-21apt07:14:422
1389199808,48cyclictest31-21ksoftirqd/107:48:531
1389199808,48cyclictest31-21ksoftirqd/107:48:531
13891998040,7cyclictest181rcu_preempt09:59:441
13891998035,16cyclictest31-21ksoftirqd/112:00:421
1389199803,31cyclictest31-21ksoftirqd/108:45:581
1389199802,44cyclictest31-21ksoftirqd/107:12:101
1389199802,44cyclictest31-21ksoftirqd/107:12:101
13883998058,11cyclictest0-21swapper/008:15:070
13883998058,11cyclictest0-21swapper/008:15:070
13883998054,17cyclictest32206-21latency_hist07:54:450
13883998038,33cyclictest4142-21cron10:19:410
13896997962,11cyclictest2374-21systemd-journal10:37:022
13896997961,9cyclictest0-21swapper/210:30:052
13896997936,9cyclictest181rcu_preempt10:10:052
1389199798,36cyclictest31-21ksoftirqd/111:14:241
1389199797,44cyclictest31-21ksoftirqd/109:29:011
1389199792,51cyclictest31-21ksoftirqd/107:28:141
1389199792,27cyclictest12080-21grep08:30:181
13883997928,9cyclictest0-21swapper/011:59:560
13896997858,13cyclictest0-21swapper/208:29:422
13896997858,10cyclictest0-21swapper/209:15:052
13896997832,8cyclictest181rcu_preempt07:05:222
13896997832,8cyclictest181rcu_preempt07:05:222
1389199786,44cyclictest31-21ksoftirqd/108:01:201
13891997857,15cyclictest12999-21latency09:40:061
1389199783,50cyclictest31-21ksoftirqd/107:44:261
13883997841,8cyclictest181rcu_preempt11:20:000
13883997841,8cyclictest181rcu_preempt11:20:000
13883997837,32cyclictest0-21swapper/009:50:380
13896997744,13cyclictest28504-21ssh11:00:022
13896997744,13cyclictest28504-21ssh11:00:022
13896997734,8cyclictest181rcu_preempt12:15:192
1389199776,34cyclictest31-21ksoftirqd/111:33:481
1389199776,34cyclictest31-21ksoftirqd/111:33:481
1389199773,44cyclictest31-21ksoftirqd/110:51:391
13883997764,9cyclictest13874-21cyclictest09:55:150
13883997757,9cyclictest0-21swapper/007:19:450
13883997757,14cyclictest17613-21latency_hist07:09:450
13883997757,14cyclictest17613-21latency_hist07:09:450
13883997753,10cyclictest181rcu_preempt11:39:590
13883997753,10cyclictest181rcu_preempt11:39:590
13896997657,9cyclictest0-21swapper/211:39:452
13896997653,14cyclictest1687-21fschecks_time08:00:042
13891997660,11cyclictest13874-21cyclictest07:50:171
13891997659,9cyclictest301ktimers/110:20:301
13891997659,9cyclictest301ktimers/110:20:301
13891997656,12cyclictest2306-21df12:04:581
13891997656,12cyclictest2306-21df12:04:581
13891997655,15cyclictest16641-21sendmail12:29:411
13891997655,14cyclictest19094-21idleruntime-cro10:44:421
13891997655,14cyclictest19094-21idleruntime-cro10:44:421
1389199764,46cyclictest31-21ksoftirqd/107:35:401
1389199764,45cyclictest31-21ksoftirqd/108:38:501
1389199763,49cyclictest31-21ksoftirqd/107:17:381
1389199763,35cyclictest31-21ksoftirqd/108:17:531
1389199763,35cyclictest31-21ksoftirqd/108:17:531
13891997612,34cyclictest31-21ksoftirqd/109:06:181
13883997660,10cyclictest23136-21ssh09:02:560
13883997660,10cyclictest23136-21ssh09:02:560
13883997650,8cyclictest181rcu_preempt10:29:470
13883997646,9cyclictest181rcu_preempt09:20:130
13883997646,9cyclictest181rcu_preempt09:20:130
13883997644,9cyclictest181rcu_preempt07:30:000
13883997635,14cyclictest181rcu_preempt09:47:200
13883997632,7cyclictest181rcu_preempt08:24:430
13896997558,9cyclictest0-21swapper/211:09:452
13896997558,9cyclictest0-21swapper/211:09:452
13896997556,9cyclictest0-21swapper/207:49:462
13896997556,9cyclictest0-21swapper/207:49:462
13896997543,23cyclictest8636-21threads08:20:162
13896997539,8cyclictest181rcu_preempt11:54:552
13896997537,9cyclictest181rcu_preempt08:54:572
13896997533,10cyclictest181rcu_preempt10:55:202
1389199758,37cyclictest31-21ksoftirqd/108:42:111
13891997554,15cyclictest7528-21sh12:14:041
13891997552,8cyclictest181rcu_preempt07:30:141
13891997552,16cyclictest20790-21diskmemload09:49:461
1389199753,40cyclictest31-21ksoftirqd/112:22:451
1389199752,30cyclictest31-21ksoftirqd/107:05:191
1389199752,30cyclictest31-21ksoftirqd/107:05:191
13891997521,25cyclictest31-21ksoftirqd/108:54:551
13883997560,8cyclictest7176-21sh09:30:100
13883997559,7cyclictest0-21swapper/011:55:040
13883997555,7cyclictest181rcu_preempt09:05:190
13883997555,14cyclictest298-21in:imuxsock10:49:450
13883997553,17cyclictest2476-21latency11:10:070
13883997548,7cyclictest181rcu_preempt09:42:150
13883997547,7cyclictest181rcu_preempt10:09:210
13896997458,7cyclictest0-21swapper/210:05:162
13896997449,19cyclictest4651-21/usr/sbin/munin08:10:102
13896997449,19cyclictest4651-21/usr/sbin/munin08:10:102
13896997437,9cyclictest181rcu_preempt09:30:542
13896997424,12cyclictest181rcu_preempt11:50:092
1389699742,28cyclictest26895-21latency_hist09:09:452
13891997459,9cyclictest31788-21kworker/u17:1-events_unbound07:55:141
13891997457,10cyclictest23926-21ssh09:04:461
13891997457,10cyclictest23926-21ssh09:04:461
1389199744,26cyclictest31-21ksoftirqd/110:55:001
1389199743,38cyclictest31-21ksoftirqd/108:05:131
1389199743,38cyclictest31-21ksoftirqd/108:05:131
1389199742,42cyclictest31-21ksoftirqd/109:18:321
13891997412,29cyclictest31-21ksoftirqd/110:13:531
13883997457,12cyclictest29680-21mailstats07:45:150
13883997457,12cyclictest29680-21mailstats07:45:150
13883997455,10cyclictest0-21swapper/010:59:440
13883997449,11cyclictest181rcu_preempt12:19:560
13883997442,7cyclictest181rcu_preempt08:29:590
13883997424,8cyclictest181rcu_preempt11:36:250
13896997359,9cyclictest7692-21ssh10:24:572
13896997326,8cyclictest181rcu_preempt11:19:432
13896997319,28cyclictest1948-21grep12:04:442
1389199733,38cyclictest31-21ksoftirqd/111:54:521
13891997325,28cyclictest31-21ksoftirqd/108:24:591
13891997315,27cyclictest31-21ksoftirqd/110:45:231
13891997313,28cyclictest31-21ksoftirqd/111:22:261
13891997313,28cyclictest31-21ksoftirqd/111:22:261
13883997353,14cyclictest19704-21fschecks_time08:55:040
13883997351,9cyclictest181rcu_preempt11:03:000
13883997351,9cyclictest181rcu_preempt11:03:000
13883997350,8cyclictest181rcu_preempt07:20:080
13883997348,16cyclictest544-21cron12:29:410
13883997334,19cyclictest23920-21latency_hist07:29:450
13896997256,10cyclictest0-21swapper/207:40:162
13896997223,12cyclictest181rcu_preempt10:19:422
1389199726,34cyclictest31-21ksoftirqd/110:00:191
1389199725,40cyclictest31-21ksoftirqd/110:15:301
1389199725,36cyclictest31-21ksoftirqd/109:10:181
1389199725,36cyclictest31-21ksoftirqd/109:10:181
13891997252,14cyclictest28262-21ntp_kernel_pll_10:05:091
13891997252,14cyclictest25883-21ssh11:49:581
13891997248,11cyclictest181rcu_preempt08:53:021
13891997248,11cyclictest181rcu_preempt08:53:021
13891997236,27cyclictest11189-21grep11:25:051
1389199722,43cyclictest31-21ksoftirqd/110:31:481
13883997258,9cyclictest15-21ksoftirqd/012:19:050
13883997252,13cyclictest15-21ksoftirqd/008:46:050
13883997250,14cyclictest15-21ksoftirqd/007:55:110
1389699717,29cyclictest39-21ksoftirqd/211:45:052
13896997112,11cyclictest39-21ksoftirqd/209:45:182
13891997152,13cyclictest19764-21sh11:39:511
13891997152,13cyclictest19764-21sh11:39:511
13891997130,12cyclictest0-21swapper/108:20:091
13891997112,33cyclictest31-21ksoftirqd/111:44:501
13883997152,7cyclictest15-21ksoftirqd/010:43:330
13883997152,7cyclictest15-21ksoftirqd/010:43:330
13883997151,10cyclictest0-21swapper/008:54:440
13883997151,10cyclictest0-21swapper/008:54:440
13883997135,13cyclictest1597-21grep08:00:020
13896997054,10cyclictest24411-21fschecks_count07:30:042
13896997041,9cyclictest181rcu_preempt12:22:502
1389199703,34cyclictest31-21ksoftirqd/109:50:361
1389199702,29cyclictest31-21ksoftirqd/109:24:461
13891997013,21cyclictest31-21ksoftirqd/112:16:531
13883997054,9cyclictest0-21swapper/010:54:450
13883997048,14cyclictest27344-21latency_hist07:39:460
13896996947,14cyclictest8764-21sh11:20:192
13896996947,14cyclictest8764-21sh11:20:192
1389699693,26cyclictest18189-21apt-get09:49:532
1389699692,38cyclictest562-21kworker/u19:0+flush-0:3210:54:412
1389699692,21cyclictest15925-21grep08:44:452
13896996911,28cyclictest18496-21sshd10:43:012
13896996911,28cyclictest18496-21sshd10:43:012
13891996950,11cyclictest181rcu_preempt09:29:551
1389199693,49cyclictest16128-21modprobe10:39:421
1389199692,33cyclictest31-21ksoftirqd/109:19:591
13883996956,7cyclictest0-21swapper/010:20:360
13883996956,7cyclictest0-21swapper/010:20:360
13883996948,8cyclictest181rcu_preempt10:04:450
13883996940,7cyclictest181rcu_preempt07:00:180
13883996938,7cyclictest181rcu_preempt08:44:450
13883996917,23cyclictest13145-21latency08:35:080
1389699685,25cyclictest0-21swapper/207:19:432
13896996849,12cyclictest20790-21diskmemload12:06:152
13896996849,12cyclictest20790-21diskmemload12:06:152
13896996837,9cyclictest181rcu_preempt07:24:552
13896996836,8cyclictest181rcu_preempt09:19:502
13896996836,8cyclictest181rcu_preempt09:19:502
13896996833,15cyclictest27959-21ssh09:10:172
13896996833,15cyclictest27959-21ssh09:10:172
13896996832,10cyclictest181rcu_preempt09:00:172
13896996832,10cyclictest181rcu_preempt09:00:172
13896996830,9cyclictest181rcu_preempt08:35:002
13896996830,9cyclictest181rcu_preempt07:55:082
1389199682,34cyclictest31-21ksoftirqd/108:10:071
1389199682,34cyclictest31-21ksoftirqd/108:10:071
1389699675,33cyclictest15317-21ssh11:31:182
1389699675,33cyclictest15317-21ssh11:31:182
13891996730,7cyclictest181rcu_preempt07:04:451
62742660,12sleep30-21swapper/309:29:423
13904996612,37cyclictest31540-21chrt11:05:003
13904996612,37cyclictest31540-21chrt11:05:003
13896996647,13cyclictest39-21ksoftirqd/209:35:122
13896996647,13cyclictest39-21ksoftirqd/209:35:122
13896996632,8cyclictest181rcu_preempt07:04:462
13896996621,10cyclictest181rcu_preempt11:44:442
13896996621,10cyclictest181rcu_preempt11:44:442
13883996647,9cyclictest0-21swapper/011:29:510
13883996647,9cyclictest0-21swapper/011:29:510
13896996534,12cyclictest17646-21kworker/u19:1-rpciod10:01:422
13883996546,7cyclictest181rcu_preempt12:13:180
13883996542,14cyclictest0-21swapper/007:10:160
13883996542,14cyclictest0-21swapper/007:10:160
1389699643,42cyclictest352-21runrttasks08:54:412
1389699643,42cyclictest352-21runrttasks08:54:412
1389699642,45cyclictest26287-21netstat07:35:112
13896996335,20cyclictest11890-21cat08:30:172
13896996213,11cyclictest4380-21apt08:09:422
13896996213,11cyclictest4380-21apt08:09:422
13883996212,12cyclictest181rcu_preempt08:29:440
1390499613,38cyclictest3202-21taskset08:05:033
1390499613,38cyclictest3202-21taskset08:05:033
1390499583,40cyclictest21131-21taskset08:59:553
1390499583,40cyclictest21131-21taskset08:59:553
1390499552,37cyclictest25300-21chrt11:49:313
1390499543,40cyclictest22418-21taskset09:55:153
1390499543,33cyclictest12018-21chrt08:30:183
1390499542,40cyclictest9555-21taskset09:34:523
1390499542,40cyclictest9555-21taskset09:34:523
1390499533,37cyclictest30710-21chrt11:57:593
1390499533,34cyclictest9498-21chrt08:24:593
1390499533,32cyclictest22671-21chrt10:50:043
1390499523,32cyclictest29683-21chrt11:01:073
1390499523,32cyclictest29683-21chrt11:01:073
1390499522,40cyclictest1225-21chrt07:59:443
1390499522,32cyclictest333-21chrt12:00:303
1390499513,34cyclictest442-21chrt09:19:453
1390499513,30cyclictest9841-21chrt10:28:433
1390499513,30cyclictest16368-21chrt08:45:453
1390499512,38cyclictest917-21kworker/3:1+events11:20:543
1390499512,38cyclictest917-21kworker/3:1+events11:20:543
1390499512,37cyclictest21805-21chrt11:42:593
1390499512,37cyclictest21805-21chrt11:42:593
1390499504,31cyclictest28054-21chrt10:05:043
1390499502,37cyclictest13389-21chrt09:40:153
1390499502,36cyclictest16128-21chrt11:33:383
1390499502,36cyclictest16128-21chrt11:33:383
1390499502,36cyclictest12669-21chrt11:27:183
1390499502,32cyclictest1216-21chrt10:14:453
1390499493,34cyclictest7750-21taskset09:30:313
1390499493,29cyclictest6360-21chrt08:14:593
1390499493,29cyclictest6360-21chrt08:14:593
1390499492,33cyclictest20036-21taskset10:45:133
1390499483,33cyclictest18263-21chrt09:49:433
1390499483,31cyclictest16856-21chrt10:40:073
1390499483,31cyclictest16856-21chrt10:40:073
1390499483,29cyclictest12737-21chrt10:33:303
1390499482,34cyclictest1012-21chrt09:20:053
1390499482,34cyclictest1012-21chrt09:20:053
1390499482,32cyclictest6237-21chrt11:15:463
1390499482,32cyclictest24856-21chrt09:05:183
1390499482,32cyclictest23227-21chrt07:25:173
1390499482,31cyclictest11907-21chrt12:20:173
125032489,23sleep30-21swapper/306:55:023
1390499473,36cyclictest917-21kworker/3:1+events10:00:353
1390499473,30cyclictest5833-21chrt10:21:063
1390499473,30cyclictest5833-21chrt10:21:063
1390499473,29cyclictest2482-21chrt11:10:073
1390499463,32cyclictest16667-21chrt12:29:413
1390499462,33cyclictest917-21kworker/3:1+events10:17:423
1390499462,29cyclictest25692-21chrt07:34:453
1390499454,31cyclictest31710-21taskset07:51:013
1390499453,31cyclictest27332-21taskset10:57:503
1390499453,30cyclictest14084-21chrt06:59:543
1390499453,29cyclictest4668-21chrt08:09:533
1390499453,29cyclictest4668-21chrt08:09:533
1390499453,28cyclictest19920-21chrt08:55:093
1390499452,33cyclictest9535-21chrt12:16:203
1390499443,32cyclictest17762-21taskset07:09:543
1390499443,32cyclictest17762-21taskset07:09:543
1390499443,29cyclictest5632-21taskset12:10:073
1390499443,29cyclictest2255-21chrt08:00:173
1390499443,28cyclictest13482-21taskset10:34:513
1390499442,32cyclictest19310-21chrt08:54:443
1390499442,32cyclictest19310-21chrt08:54:443
1390499442,31cyclictest19500-21taskset11:39:423
1390499442,29cyclictest28932-21taskset07:44:453
1390499442,27cyclictest12849-21taskset08:35:003
1390499442,25cyclictest28041-21chrt11:53:493
1390499433,29cyclictest2177-21chrt12:04:563
1390499433,29cyclictest2177-21chrt12:04:563
1390499433,28cyclictest15965-21taskset07:04:523
1390499433,28cyclictest15965-21taskset07:04:523
1390499423,29cyclictest29972-21chrt07:45:183
1390499423,29cyclictest29972-21chrt07:45:183
1390499423,29cyclictest26046-21taskset07:35:063
1390499419,24cyclictest18847-21taskset09:50:083
1390499413,28cyclictest7860-21taskset08:19:493
1390499412,27cyclictest19369-21taskset07:14:573
1390499403,27cyclictest14692-21chrt08:40:063
1390499403,25cyclictest27047-21taskset09:09:483
1390499403,25cyclictest27047-21taskset09:09:483
1390499403,25cyclictest20944-21taskset07:19:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional