You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 09:55
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Sat May 23, 2026 00:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48759910363,34cyclictest2374-21systemd-journal22:27:342
540921000,8sleep10-21swapper/122:32:051
4867999960,31cyclictest4852-21cyclictest21:20:101
4867999960,31cyclictest4852-21cyclictest21:20:101
4867999855,36cyclictest0-21swapper/121:25:581
34082989,59sleep23485-21awk18:55:562
4867999631,13cyclictest0-21swapper/119:30:421
486799962,36cyclictest11318-21latency_hist20:55:401
486799962,36cyclictest11318-21latency_hist20:55:401
486799953,41cyclictest10820-21timerandwakeup22:41:121
486799952,34cyclictest1481-21memory22:26:031
487599932,61cyclictest21455-21apt-get22:06:112
487599932,61cyclictest21455-21apt-get22:06:112
4867999338,8cyclictest0-21swapper/120:15:411
4867999332,9cyclictest0-21swapper/123:55:281
4867999327,11cyclictest0-21swapper/119:20:411
4867999327,11cyclictest0-21swapper/119:20:411
486799932,36cyclictest7793-21cpu00:25:521
486799932,32cyclictest27426-21apt-get20:06:101
487599923,64cyclictest28120-21ssh23:10:562
487599923,64cyclictest28120-21ssh23:10:562
487599922,43cyclictest11386-21apt-get20:55:482
486799923,40cyclictest23903-21ssh22:10:061
486799923,40cyclictest23903-21ssh22:10:061
486799923,35cyclictest5710-21ls23:27:291
486799923,35cyclictest25037-21/usr/sbin/munin21:20:481
486799923,33cyclictest25901-21taskset00:01:071
4867999228,14cyclictest0-21swapper/121:50:591
4875999168,15cyclictest21646-21unixbench_multi19:46:132
4875999168,15cyclictest21646-21unixbench_multi19:46:132
4875999150,35cyclictest10868-21timerwakeupswit22:41:122
4867999138,11cyclictest0-21swapper/123:00:521
4867999136,8cyclictest0-21swapper/100:15:361
4867999136,8cyclictest0-21swapper/100:15:361
486799913,34cyclictest15867-21ps21:56:101
486799913,34cyclictest15867-21ps21:56:101
486799913,33cyclictest13949-21ssh23:41:071
486799912,37cyclictest7717-21fschecks_time23:30:591
486799903,34cyclictest28597-21fschecks_time23:11:381
486799903,34cyclictest28597-21fschecks_time23:11:381
486799902,36cyclictest8747-21apt-key19:10:411
487599897,35cyclictest13582-21needreboot21:01:042
487599897,35cyclictest13582-21needreboot21:01:042
4875998948,33cyclictest0-21swapper/222:35:592
4875998948,33cyclictest0-21swapper/222:35:592
487599893,37cyclictest26782-21ssh21:22:192
4875998931,43cyclictest0-21swapper/200:21:052
4875998931,43cyclictest0-21swapper/200:21:052
4875998927,50cyclictest0-21swapper/222:18:422
4875998915,32cyclictest15010-21grep19:26:572
486799894,33cyclictest2849-21latency_hist20:30:391
4867998934,9cyclictest0-21swapper/120:41:441
486799893,36cyclictest27039-21gpgv20:05:401
486799893,35cyclictest28118-21grep22:16:081
487599884,35cyclictest31394-21cstates00:10:542
487599884,35cyclictest31394-21cstates00:10:542
487599883,29cyclictest1569150cat22:50:392
486799883,39cyclictest3079-21latency_hist21:35:401
486799883,35cyclictest9124-21vmstat20:46:141
486799882,37cyclictest13910-21rm22:46:131
486799882,35cyclictest24318-21runrttasks23:58:501
486799882,34cyclictest1230-21latency_hist23:20:391
486799874,38cyclictest20981-21fschecks_time19:45:581
486799874,38cyclictest20981-21fschecks_time19:45:581
486799873,35cyclictest9961-21unixbench_singl19:11:371
486799873,35cyclictest4948-21latency_hist19:00:421
486799873,35cyclictest1369-21latency_hist20:25:401
486799873,35cyclictest1369-21latency_hist20:25:401
486799872,37cyclictest20623-21latency_hist19:45:401
487599864,36cyclictest4699-21/usr/sbin/munin23:27:292
487599863,38cyclictest12442-21grep21:50:592
487599863,34cyclictest352-21runrttasks00:27:202
4875998633,43cyclictest0-21swapper/221:36:062
486799864,33cyclictest24704-21ssh22:10:591
486799864,33cyclictest1523-21/usr/sbin/munin23:20:591
4867998638,11cyclictest4852-21cyclictest00:16:231
4867998636,10cyclictest12733-21diskmemload22:02:511
486799863,35cyclictest18468-21unixbench_singl19:36:121
486799863,33cyclictest16293-21grep21:05:591
486799863,33cyclictest16293-21grep21:05:591
486799863,31cyclictest30713-21fschecks_time22:21:001
4867998630,11cyclictest0-21swapper/121:13:111
486799862,37cyclictest352-21runrttasks21:40:421
4875998563,16cyclictest8441-21/usr/sbin/munin19:10:392
487599853,38cyclictest11015-21/usr/sbin/munin23:36:082
487599853,31cyclictest19353-21sh23:50:522
487599852,21cyclictest956-21expr21:31:092
4875998514,30cyclictest22766-21grep23:01:122
486799855,37cyclictest4458-21latency_hist20:35:391
486799853,35cyclictest27780-21cut23:10:401
486799853,35cyclictest27780-21cut23:10:401
486799853,33cyclictest352-21runrttasks00:06:431
486799853,33cyclictest15904-21/usr/sbin/munin22:50:571
486799853,33cyclictest12110-21switchtime20:56:111
4861998570,9cyclictest28187-21ssh23:11:070
4861998570,9cyclictest28187-21ssh23:11:070
4861998560,10cyclictest181rcu_preempt23:29:360
4861998551,13cyclictest181rcu_preempt21:34:150
487599845,37cyclictest19274-21if_err_eth022:56:002
487599843,37cyclictest13945-21ntpq23:41:062
487599843,24cyclictest6088-21unin-run20:40:382
487599843,24cyclictest6088-21unin-run20:40:382
4875998417,27cyclictest3836-21diskmemload00:18:562
4875998363,12cyclictest0-21swapper/219:21:322
4875998363,12cyclictest0-21swapper/219:21:322
487599834,35cyclictest24813-21/usr/sbin/munin23:06:132
487599834,35cyclictest24813-21/usr/sbin/munin23:06:132
487599833,34cyclictest28240-21/usr/sbin/munin00:06:032
487599833,34cyclictest23713-21latency_hist19:55:412
487599833,34cyclictest16723-21timerandwakeup19:31:272
487599833,31cyclictest17939-21runrttasks22:53:482
487599833,28cyclictest352-21runrttasks20:47:162
487599833,26cyclictest15177-21/usr/sbin/munin21:56:012
487599833,26cyclictest15177-21/usr/sbin/munin21:56:012
487599833,20cyclictest352-21runrttasks20:05:132
487599832,26cyclictest18154-21/usr/sbin/munin22:01:112
486799833,35cyclictest4728-21/usr/sbin/munin00:20:551
486799833,35cyclictest4728-21/usr/sbin/munin00:20:551
486799833,32cyclictest10355-21/usr/sbin/munin23:36:111
4861998358,19cyclictest4983-21latency_hist19:00:420
4861998325,32cyclictest181rcu_preempt19:35:370
487599823,33cyclictest352-21runrttasks21:08:272
487599823,33cyclictest352-21runrttasks21:08:272
487599823,27cyclictest24418-21/usr/sbin/munin22:11:012
4875998215,29cyclictest21970-21sort21:15:402
486799823,35cyclictest25421-21latency_hist20:00:391
4861998265,11cyclictest2112-21ntp_kernel_pll_23:21:050
4875998160,15cyclictest31049-21sh21:27:132
487599813,24cyclictest4328-21latency_hist23:25:392
487599813,23cyclictest1229-21cron23:20:362
487599813,22cyclictest29830-21/usr/sbin/munin20:11:092
4875998115,29cyclictest30457-21sh22:20:532
486799813,34cyclictest12670-21grep19:21:561
486799813,34cyclictest12670-21grep19:21:561
4861998160,14cyclictest21651-21latency_hist23:00:390
4861998154,20cyclictest31255-21fschecks_count23:15:570
487599804,33cyclictest24549-21proc_pri19:56:102
487599803,59cyclictest27426-21apt-get20:05:532
487599803,38cyclictest23764-21ssh21:17:552
487599803,38cyclictest23764-21ssh21:17:552
487599803,34cyclictest10065-21latency_hist23:35:362
487599803,23cyclictest6031-21apt-get20:40:512
487599802,25cyclictest5959-21taskset19:01:132
487599802,25cyclictest5959-21taskset19:01:132
486799803,33cyclictest7313-21ssh22:35:591
486799803,33cyclictest7313-21ssh22:35:591
4861998050,21cyclictest0-21swapper/022:06:400
4861998050,21cyclictest0-21swapper/022:06:400
4861998037,13cyclictest181rcu_preempt21:49:310
487599794,35cyclictest10776-21fschecks_count19:15:582
487599794,35cyclictest10776-21fschecks_count19:15:582
487599794,30cyclictest352-21runrttasks20:16:552
487599794,30cyclictest352-21runrttasks20:16:552
487599794,27cyclictest9419-21cut21:45:592
487599793,23cyclictest3327-21fschecks_time20:30:592
487599792,68cyclictest8436-21apt-get19:10:522
487599792,26cyclictest18444-21/usr/sbin/munin19:36:122
487599792,25cyclictest2019-21wc20:26:072
487599792,24cyclictest17380-21head23:46:132
487599792,24cyclictest17380-21head23:46:132
4861997960,14cyclictest58-21kcompactd020:51:010
4861997960,14cyclictest58-21kcompactd020:51:010
4861997959,13cyclictest58-21kcompactd023:35:580
4861997959,12cyclictest5369-21ssh00:21:060
4861997959,12cyclictest5369-21ssh00:21:060
4861997957,16cyclictest25402-21ntp_kernel_pll_23:06:040
4861997957,16cyclictest25402-21ntp_kernel_pll_23:06:040
488299783,63cyclictest4802-21chrt23:25:593
4875997855,15cyclictest12733-21diskmemload21:41:012
487599783,29cyclictest352-21runrttasks22:33:012
487599783,27cyclictest22495-21runrttasks23:55:562
487599783,24cyclictest25892-21ntp_states00:01:072
487599783,20cyclictest352-21runrttasks20:23:022
487599783,20cyclictest352-21runrttasks20:23:022
486799783,32cyclictest19121-21grep22:55:581
4861997839,13cyclictest181rcu_preempt00:01:590
4861997831,16cyclictest181rcu_preempt22:11:030
487599774,34cyclictest9863-21cstates20:50:552
487599774,34cyclictest9863-21cstates20:50:552
487599772,34cyclictest20695-21latency_hist19:45:412
486199777,40cyclictest15-21ksoftirqd/021:35:490
4861997761,10cyclictest2843-21systemd-run20:30:380
4861997759,12cyclictest12733-21diskmemload21:00:580
4861997759,12cyclictest12733-21diskmemload21:00:580
486199774,44cyclictest352-21runrttasks21:19:140
486199774,44cyclictest352-21runrttasks21:19:140
4861997734,16cyclictest181rcu_preempt21:07:450
4861997734,16cyclictest181rcu_preempt21:07:450
4861997731,8cyclictest181rcu_preempt21:56:130
4861997731,8cyclictest181rcu_preempt21:56:130
4861997721,30cyclictest181rcu_preempt22:46:210
4861997655,15cyclictest23084-21switchtime19:51:110
4861997653,14cyclictest12733-21diskmemload23:55:400
4861997650,9cyclictest181rcu_preempt22:01:100
4861997646,9cyclictest181rcu_preempt20:13:130
486199763,46cyclictest12733-21diskmemload22:24:210
488299755,52cyclictest17246-21chrt19:35:373
4861997555,14cyclictest15447-21apt-config19:30:390
4861997531,17cyclictest181rcu_preempt20:41:360
488299744,51cyclictest0-21swapper/320:40:593
4867997455,13cyclictest22463-21cstates19:50:541
4861997447,9cyclictest181rcu_preempt23:48:420
4861997447,9cyclictest181rcu_preempt23:48:420
4861997445,14cyclictest181rcu_preempt22:32:180
486199744,39cyclictest1608-21latency_hist00:15:390
486199744,39cyclictest1608-21latency_hist00:15:390
486199743,57cyclictest1199-21fschecks_count22:25:580
486199742,39cyclictest15-21ksoftirqd/000:18:100
210972740,6sleep30-21swapper/322:05:403
4861997359,9cyclictest9915-21ssh23:35:140
4861997355,12cyclictest21568-21kworker/u17:0+nfsiod00:28:580
4861997351,15cyclictest6883-21ssh21:41:100
4861997348,8cyclictest181rcu_preempt19:45:490
4861997348,8cyclictest181rcu_preempt19:45:490
4861997340,12cyclictest181rcu_preempt23:58:100
4861997311,24cyclictest15-21ksoftirqd/023:02:560
4882997231,25cyclictest31323-21taskset23:15:573
4867997256,10cyclictest15149-21diskmemload21:04:011
4867997256,10cyclictest15149-21diskmemload21:04:011
4867997253,13cyclictest16559-21fschecks_count23:45:571
4867997253,13cyclictest16559-21fschecks_count23:45:571
4861997246,11cyclictest181rcu_preempt22:54:530
4861997244,20cyclictest0-21swapper/021:22:310
4861997240,9cyclictest181rcu_preempt19:45:400
4861997239,13cyclictest181rcu_preempt23:45:400
4861997238,12cyclictest181rcu_preempt20:35:390
4861997231,18cyclictest181rcu_preempt20:06:220
4861997152,14cyclictest29391-21ssh22:18:360
4861997152,11cyclictest19941-21ssh21:11:140
4861997151,13cyclictest4852-21cyclictest20:50:410
4861997149,15cyclictest11915-21apt-get19:21:180
4861997149,15cyclictest11915-21apt-get19:21:180
486199714,54cyclictest19676-21snmpd20:15:530
486199714,54cyclictest19676-21snmpd20:15:530
486199714,34cyclictest28240-21/usr/sbin/munin00:05:550
486199713,57cyclictest10260-21fschecks_time22:40:590
4882997031,25cyclictest5528-21taskset20:36:123
4882997031,25cyclictest5528-21taskset20:36:123
4861997045,17cyclictest6030-21latency_hist20:40:390
4861997045,17cyclictest6030-21latency_hist20:40:390
4861997044,20cyclictest4852-21cyclictest20:05:400
486199704,38cyclictest352-21runrttasks21:54:170
4861997033,25cyclictest25311-21modprobe20:00:370
4882996930,26cyclictest25234-21taskset23:06:003
4882996930,26cyclictest25234-21taskset23:06:003
4867996951,11cyclictest11085-21ntpd21:36:091
486199698,28cyclictest15-21ksoftirqd/020:20:510
486199698,28cyclictest15-21ksoftirqd/020:20:510
4861996937,11cyclictest181rcu_preempt19:16:300
4861996937,11cyclictest181rcu_preempt19:16:300
486199693,44cyclictest6638-21kworker/0:0+pm19:10:400
4861996927,13cyclictest181rcu_preempt21:27:390
4882996831,23cyclictest31664-21taskset20:16:153
4882996831,23cyclictest31664-21taskset20:16:153
4882996828,29cyclictest23678-21chrt23:57:013
4882996828,27cyclictest28426-21taskset23:11:233
4882996828,27cyclictest28426-21taskset23:11:233
4882996828,25cyclictest3722-21taskset00:18:383
4867996851,11cyclictest277-21dbus-daemon21:50:391
486199689,14cyclictest17924-21cat19:36:010
4882996728,24cyclictest23243-21taskset21:16:263
4882996728,24cyclictest23243-21taskset21:16:263
486199674,34cyclictest12735-21latency_hist21:00:390
4861996743,11cyclictest181rcu_preempt22:38:460
4861996743,11cyclictest181rcu_preempt22:38:460
486199672,54cyclictest352-21runrttasks19:00:590
486199672,54cyclictest352-21runrttasks19:00:590
4861996714,34cyclictest10460-21date19:15:400
4882996632,21cyclictest32382-21chrt00:11:163
4882996632,21cyclictest32382-21chrt00:11:163
4882996631,21cyclictest12701-21chrt23:39:423
4882996629,25cyclictest11079-21taskset19:16:043
4882996629,25cyclictest11079-21taskset19:16:043
4882996628,24cyclictest28600-21taskset20:08:263
4882996628,24cyclictest27347-21taskset00:03:583
4882996626,26cyclictest17672-21taskset22:53:033
4882996610,42cyclictest0-21swapper/322:06:043
4882996610,42cyclictest0-21swapper/322:06:043
4867996650,10cyclictest32026-21grep20:20:371
4867996650,10cyclictest32026-21grep20:20:371
4867996643,17cyclictest17244-21apt19:35:371
4882996529,24cyclictest3292-21taskset22:29:363
4882996528,25cyclictest19016-21chrt23:50:363
4882996528,25cyclictest19016-21chrt23:50:363
4882996528,24cyclictest8699-21chrt20:46:083
4882996528,24cyclictest15214-21chrt19:28:443
4882996527,25cyclictest32120-21taskset21:30:133
4882996527,25cyclictest20169-21taskset19:42:013
4882996527,24cyclictest14150-21taskset21:01:143
4882996527,24cyclictest14150-21taskset21:01:143
4882996526,26cyclictest9883-21taskset23:35:083
4882996525,26cyclictest27469-21taskset22:15:503
4882996430,22cyclictest8072-21chrt21:43:263
4882996430,21cyclictest25593-21chrt20:00:523
4882996430,20cyclictest21404-21chrt19:46:103
4882996430,20cyclictest21404-21chrt19:46:103
4882996427,23cyclictest26703-21taskset22:14:243
4882996331,21cyclictest3351-21chrt21:35:573
4882996330,20cyclictest12995-21chrt19:22:413
4882996330,20cyclictest12995-21chrt19:22:413
4882996329,21cyclictest428-21chrt20:21:093
4882996329,21cyclictest428-21chrt20:21:093
4882996326,25cyclictest13381-21taskset23:40:513
4882996230,21cyclictest21097-21chrt23:53:083
4882996230,20cyclictest12155-21chrt21:50:443
4882996229,21cyclictest8103-21chrt00:25:593
4882996229,21cyclictest26319-21chrt21:21:413
4882996228,22cyclictest8123-21chrt19:06:533
4882996228,21cyclictest10921-21chrt20:53:123
4882996228,21cyclictest10921-21chrt20:53:123
4882996228,20cyclictest7915-21chrt22:36:133
4882996228,20cyclictest7915-21chrt22:36:133
4882996227,23cyclictest30634-21taskset00:09:503
4882996225,23cyclictest13045-21taskset22:45:533
4867996245,11cyclictest352-21runrttasks19:01:131
4867996245,11cyclictest352-21runrttasks19:01:131
486799622,44cyclictest6416-21cat20:40:411
486799622,44cyclictest6416-21cat20:40:411
4882996131,19cyclictest16355-21chrt21:56:243
4882996131,19cyclictest16355-21chrt21:56:243
4882996130,19cyclictest24147-21chrt23:04:343
4882996129,20cyclictest30063-21chrt20:11:133
4882996129,20cyclictest1923-21chrt23:21:003
4882996129,19cyclictest5161-21chrt00:21:013
4882996129,19cyclictest5161-21chrt00:21:013
4882996128,21cyclictest1585-21chrt21:32:073
4882996128,20cyclictest11172-21taskset22:41:363
4882996127,23cyclictest11394-21chrt20:55:513
4882996127,20cyclictest21044-21chrt22:59:113
4882996126,24cyclictest31634-21taskset22:21:333
4882996126,21cyclictest4136-21chrt22:30:533
4882996029,20cyclictest8942-21chrt19:10:463
4882996028,21cyclictest9143-21chrt21:45:453
4882996028,21cyclictest18594-21chrt19:37:273
4882996028,19cyclictest19093-21chrt21:10:513
4882996027,21cyclictest6196-21chrt19:03:173
4882996027,21cyclictest6196-21chrt19:03:173
4882996027,21cyclictest3607-21chrt20:31:053
4882996026,22cyclictest23451-21chrt19:52:263
4882995928,20cyclictest16595-21chrt21:06:053
4882995928,20cyclictest16595-21chrt21:06:053
4882995927,20cyclictest24996-21chrt19:56:393
4882995926,20cyclictest2459-21chrt20:26:363
348625918,25sleep30-21swapper/318:55:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional