You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 08:49
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sat Mar 21, 2026 00:44:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1035521150,8sleep010359-21cat22:19:540
1035521150,8sleep010359-21cat22:19:540
1618621040,8sleep20-21swapper/223:24:372
281642990,6sleep30-21swapper/319:19:543
22502999477,9cyclictest0-21swapper/123:29:531
22497999169,14cyclictest7254-21kworker/u17:019:54:490
22497999169,14cyclictest7254-21kworker/u17:019:54:490
22502999071,11cyclictest297-21systemd-logind19:09:531
22497999064,17cyclictest14030-21apt-key20:14:540
2249799898,72cyclictest22864-21modprobe23:35:120
2249799898,72cyclictest22864-21modprobe23:35:120
22502998870,9cyclictest0-21swapper/121:40:201
22502998864,18cyclictest12682-21latency_hist21:29:521
221182876,61sleep10-21swapper/119:01:321
2252299865,50cyclictest0-21swapper/323:54:513
2252299865,50cyclictest0-21swapper/323:54:513
22514998662,15cyclictest30134-21diskmemload22:40:062
22514998662,15cyclictest30134-21diskmemload22:40:062
22497998659,19cyclictest18630-21timerandwakeup00:21:170
22514998566,13cyclictest2480-21ssh23:00:222
22514998561,17cyclictest4056-21cut19:44:522
22497998560,17cyclictest22519-21latency_hist19:04:530
22514998463,15cyclictest26197-21latency_hist00:34:522
22502998438,37cyclictest10128-21grep20:00:271
22497998465,13cyclictest21590-21ssh21:44:500
22497998464,14cyclictest297-21systemd-logind20:54:500
22514998360,15cyclictest25458-21sh23:39:492
22514998360,15cyclictest25458-21sh23:39:492
22514998359,11cyclictest181rcu_preempt19:19:532
2249799833,62cyclictest2345-21ntp_states23:00:190
22502998261,14cyclictest3682-21kworker/u17:2+events_unbound22:34:491
22502998261,14cyclictest3682-21kworker/u17:2+events_unbound22:34:491
2250299823,64cyclictest19872-21ssh23:30:221
2250299823,64cyclictest19872-21ssh23:30:221
22497998251,13cyclictest181rcu_preempt22:04:130
22502998148,24cyclictest614-21kworker/u18:2+nfsiod21:19:261
22497998147,25cyclictest0-21swapper/022:35:100
14742810,8sleep01478-21sendmail_mailqu19:35:220
14742810,8sleep01478-21sendmail_mailqu19:35:220
22514998063,11cyclictest22488-21cyclictest23:55:222
22502998062,11cyclictest0-21swapper/120:19:491
22497998041,13cyclictest181rcu_preempt22:19:560
2249799803,38cyclictest19981-21sshd23:31:070
2249799803,38cyclictest19981-21sshd23:31:070
22497998026,34cyclictest31520-21grep21:05:260
2250299793,67cyclictest17399-21apt-get00:20:081
22497997955,19cyclictest8522-21kworker/u18:0+events_unbound22:59:490
22497997954,17cyclictest22488-21cyclictest22:44:530
22497997954,17cyclictest22488-21cyclictest22:44:530
22497997940,16cyclictest181rcu_preempt20:46:350
22497997940,16cyclictest181rcu_preempt20:46:350
18902790,7sleep30-21swapper/322:05:183
18902790,7sleep30-21swapper/322:05:183
2249799789,56cyclictest15599-21apt20:19:490
2249799785,63cyclictest31409-21apt23:49:490
22497997849,19cyclictest6329-21sh00:00:270
22497997849,19cyclictest6329-21sh00:00:270
2249799783,36cyclictest17334-21date00:19:530
22522997736,24cyclictest22321-21chrt23:34:503
22522997736,24cyclictest22321-21chrt23:34:503
22514997763,10cyclictest22488-21cyclictest22:15:202
22514997763,10cyclictest22488-21cyclictest22:15:202
22514997761,11cyclictest4841-21ntp_offset19:45:192
22514997758,13cyclictest16855-21latency_hist00:19:532
22514997752,12cyclictest181rcu_preempt21:04:522
22502997759,9cyclictest0-21swapper/120:25:251
22502997757,14cyclictest29716-21latency_hist19:24:521
22502997757,14cyclictest29716-21latency_hist19:24:521
2250299773,64cyclictest10563-21ssh22:20:111
22497997754,16cyclictest30134-21diskmemload23:44:510
22497997727,17cyclictest181rcu_preempt00:30:410
2249799772,26cyclictest22574-21tune2fs20:40:110
2249799772,26cyclictest22574-21tune2fs20:40:110
22497997715,41cyclictest31408-21modprobe19:29:490
22497997715,41cyclictest31408-21modprobe19:29:490
22514997658,12cyclictest22310-21kernelversion21:45:172
22514997650,20cyclictest13642-21sh22:25:102
22514997617,36cyclictest23544-21timerwakeupswit19:05:232
22502997655,14cyclictest25295-21systemd-run20:49:501
22502997655,14cyclictest25295-21systemd-run20:49:501
22502997655,14cyclictest1-21systemd20:20:071
22502997650,17cyclictest26102-21sh22:45:251
22497997651,18cyclictest30134-21diskmemload21:55:130
2249799763,35cyclictest352-21runrttasks23:26:500
22514997557,12cyclictest12211-21unixbench_multi00:10:262
22514997556,12cyclictest19741-21idleruntime22:35:112
22514997555,14cyclictest6750-21latency_hist21:19:512
22514997550,19cyclictest13132-21munin-plugin-st22:24:492
22502997550,18cyclictest16084-21latency_hist22:29:521
22502997542,19cyclictest23653-21modprobe20:44:491
22502997542,19cyclictest23653-21modprobe20:44:491
22497997541,12cyclictest181rcu_preempt21:35:180
22497997535,7cyclictest181rcu_preempt22:10:260
22497997532,17cyclictest6854-21latency_hist21:19:520
22514997451,13cyclictest0-21swapper/223:29:542
22514997446,21cyclictest17211-21cut20:24:522
22502997456,12cyclictest7878-21if_eth019:55:131
2250299742,61cyclictest22809-21apt-get23:35:111
2250299742,61cyclictest22809-21apt-get23:35:111
22497997455,12cyclictest12822-21ssh21:29:520
22497997440,12cyclictest181rcu_preempt22:09:000
22497997440,12cyclictest181rcu_preempt22:09:000
22497997425,30cyclictest2703-21apt-get19:40:140
2249799742,36cyclictest7838-21switchtime21:20:240
2249799742,36cyclictest7838-21switchtime21:20:240
22497997421,9cyclictest181rcu_preempt20:30:250
22497997416,14cyclictest181rcu_preempt20:00:260
22514997357,11cyclictest22488-21cyclictest19:50:162
22514997357,11cyclictest22488-21cyclictest19:50:162
22514997355,13cyclictest22488-21cyclictest21:44:482
22514997347,20cyclictest22226-21latency_hist23:34:542
22514997347,20cyclictest22226-21latency_hist23:34:542
2251499732,27cyclictest20542-21latency_hist20:34:522
22502997355,11cyclictest28540-21systemd-run20:59:501
22502997350,15cyclictest30050-21/usr/sbin/munin19:25:341
22502997350,15cyclictest30050-21/usr/sbin/munin19:25:341
22502997345,18cyclictest14088-21latency_hist20:14:541
22502997329,31cyclictest14505-21cut00:15:111
22502997321,36cyclictest19086-21sshd20:30:061
2249799735,38cyclictest4940-21modprobe23:59:490
22497997346,9cyclictest181rcu_preempt20:05:050
22497997344,10cyclictest181rcu_preempt21:45:070
2249799734,29cyclictest27307-21ssh22:48:020
22497997334,10cyclictest181rcu_preempt19:21:130
22497997334,10cyclictest181rcu_preempt19:21:130
22522997233,26cyclictest31057-21taskset21:05:183
22522997232,27cyclictest4919-21taskset22:10:203
22514997257,9cyclictest9978-21sh23:13:532
22514997257,9cyclictest9978-21sh23:13:532
22514997257,11cyclictest22488-21cyclictest22:47:372
22514997248,18cyclictest30134-21diskmemload22:34:522
22514997248,18cyclictest30134-21diskmemload22:34:522
22514997217,14cyclictest181rcu_preempt00:05:112
22497997252,15cyclictest22488-21cyclictest00:14:530
22497997248,7cyclictest181rcu_preempt21:50:050
22497997245,18cyclictest0-21swapper/022:50:110
22497997245,18cyclictest0-21swapper/022:50:110
22497997243,21cyclictest0-21swapper/020:35:000
22497997243,21cyclictest0-21swapper/020:35:000
2249799723,33cyclictest28103-21latency_hist19:19:530
22514997152,10cyclictest0-21swapper/223:45:212
22514997151,14cyclictest30134-21diskmemload00:24:492
22514997151,14cyclictest16369-21kernelversion21:35:142
22514997150,15cyclictest352-21runrttasks20:15:542
22502997154,10cyclictest22488-21cyclictest21:33:431
22502997154,10cyclictest22488-21cyclictest21:33:431
22502997153,11cyclictest4556-21ssh22:10:111
2250299713,46cyclictest32259-21ssh22:55:261
22502997113,27cyclictest0-21swapper/123:05:051
22497997148,13cyclictest0-21swapper/022:24:590
22497997131,11cyclictest30108-21cron21:04:490
22497997130,8cyclictest181rcu_preempt00:29:490
22497997130,8cyclictest181rcu_preempt00:29:490
22522997036,22cyclictest7593-21chrt22:15:113
22522997036,22cyclictest7593-21chrt22:15:113
22522997032,24cyclictest23031-21taskset19:05:123
2251499705,27cyclictest39-21ksoftirqd/221:57:112
22514997052,11cyclictest7010-21sh22:14:502
22514997052,11cyclictest1-21systemd19:13:382
22514997051,11cyclictest0-21swapper/220:45:102
22514997051,11cyclictest0-21swapper/220:45:102
22514997048,11cyclictest0-21swapper/220:40:122
22514997048,11cyclictest0-21swapper/220:40:122
22514997047,14cyclictest31385-21ntp_kernel_err22:00:182
22514997046,17cyclictest2335-21unixbench_multi22:05:272
22514997046,17cyclictest2335-21unixbench_multi22:05:272
22514997046,15cyclictest30134-21diskmemload00:25:082
22514997046,15cyclictest30134-21diskmemload00:25:082
22514997045,19cyclictest29186-21sh22:50:262
22514997045,19cyclictest29186-21sh22:50:262
2250299706,51cyclictest2167-21cat23:54:541
2250299706,51cyclictest2167-21cat23:54:541
22502997055,10cyclictest22488-21cyclictest23:24:491
22502997050,14cyclictest19676-21snmpd22:50:221
22502997050,14cyclictest19676-21snmpd22:50:221
22502997048,12cyclictest0-21swapper/121:54:491
22502997047,13cyclictest0-21swapper/123:00:161
22497997026,12cyclictest0-21swapper/021:14:520
22497997026,12cyclictest0-21swapper/021:14:520
22522996936,22cyclictest29764-21taskset19:24:523
22522996936,22cyclictest29764-21taskset19:24:523
22522996932,25cyclictest23809-21taskset21:47:473
22514996955,8cyclictest0-21swapper/221:30:282
22514996955,8cyclictest0-21swapper/221:30:282
22514996941,20cyclictest1208-21ssh21:10:112
22514996941,20cyclictest1208-21ssh21:10:112
22497996950,12cyclictest22488-21cyclictest23:16:010
22497996934,26cyclictest1997-40mandb23:54:520
22497996934,26cyclictest1997-40mandb23:54:520
22497996928,30cyclictest0-21swapper/021:34:360
22497996928,30cyclictest0-21swapper/021:34:360
131242690,6sleep10-21swapper/100:12:471
22522996835,22cyclictest10523-21chrt22:20:063
22522996835,21cyclictest5709-21chrt21:17:063
22522996834,21cyclictest31727-21taskset22:55:153
22522996834,21cyclictest19605-21chrt20:30:193
22522996833,22cyclictest25972-21chrt23:40:103
22522996831,25cyclictest11396-21chrt00:10:093
22522996831,23cyclictest7416-21taskset21:20:143
22522996831,23cyclictest7416-21taskset21:20:143
22514996853,8cyclictest0-21swapper/220:55:262
22514996852,9cyclictest0-21swapper/223:15:152
22514996851,11cyclictest6984-21/usr/sbin/munin21:20:172
22514996851,11cyclictest6984-21/usr/sbin/munin21:20:172
22514996851,10cyclictest0-21swapper/219:39:492
22514996851,10cyclictest0-21swapper/219:39:492
22514996850,9cyclictest0-21swapper/220:05:222
22514996849,9cyclictest0-21swapper/200:00:222
22514996849,9cyclictest0-21swapper/200:00:222
22514996847,15cyclictest300-21rs:main2
2251499682,43cyclictest30134-21diskmemload23:54:512
2251499682,43cyclictest30134-21diskmemload23:54:512
2251499682,28cyclictest9429-21fschecks_time20:00:132
22514996821,37cyclictest7765-21fschecks_count19:55:112
22502996850,11cyclictest277-21dbus-daemon21:59:501
2250299684,41cyclictest22179-21latency_hist20:39:521
2250299684,41cyclictest22179-21latency_hist20:39:521
22502996842,17cyclictest2502-21fschecks_count23:55:111
22502996840,20cyclictest28413-21cron23:44:481
2250299683,44cyclictest11026-21fschecks_time20:05:111
22497996853,8cyclictest9654-21rm00:06:210
22497996840,10cyclictest181rcu_preempt20:25:210
22497996839,12cyclictest181rcu_preempt23:10:240
22497996839,12cyclictest181rcu_preempt23:10:240
22497996839,11cyclictest181rcu_preempt19:10:080
22497996838,12cyclictest181rcu_preempt23:09:490
2249799682,47cyclictest14194-21ntp_kernel_pll_23:20:180
2242026810,38sleep222433-21systemd-run19:04:512
22522996734,22cyclictest16029-21chrt20:20:113
22522996732,22cyclictest21160-21chrt21:43:413
22522996731,23cyclictest9423-21taskset23:12:183
22522996731,23cyclictest9423-21taskset23:12:183
22522996731,22cyclictest18278-21chrt22:32:373
22522996731,22cyclictest18278-21chrt22:32:373
22514996744,16cyclictest22488-21cyclictest19:21:212
22514996744,16cyclictest22488-21cyclictest19:21:212
2251499674,38cyclictest30422-21/usr/sbin/munin21:05:092
22502996752,8cyclictest0-21swapper/121:20:261
22502996752,8cyclictest0-21swapper/121:20:261
22502996751,11cyclictest7989-21latency_hist00:04:511
22502996751,11cyclictest7989-21latency_hist00:04:511
22502996745,14cyclictest22488-21cyclictest21:36:551
2250299673,43cyclictest10900-21fschecks_count23:15:111
22497996746,14cyclictest0-21swapper/020:59:490
22497996745,15cyclictest22488-21cyclictest19:30:230
2249799673,46cyclictest17382-21latency_hist20:24:530
22497996734,12cyclictest181rcu_preempt22:30:230
22497996734,12cyclictest181rcu_preempt22:30:230
22497996731,8cyclictest181rcu_preempt19:45:170
22497996731,12cyclictest181rcu_preempt19:05:020
22522996634,21cyclictest17352-21chrt00:19:543
22522996634,20cyclictest23940-21taskset00:30:163
22522996633,23cyclictest30067-21chrt19:25:003
22522996633,23cyclictest30067-21chrt19:25:003
22522996633,21cyclictest15312-21chrt22:27:383
22522996633,20cyclictest21535-21chrt00:25:343
22522996633,20cyclictest21535-21chrt00:25:343
22522996631,24cyclictest14763-21chrt00:15:163
22522996631,22cyclictest1749-21chrt23:00:013
22522996630,24cyclictest29956-21taskset21:58:163
22522996630,23cyclictest28221-21taskset20:56:113
22522996627,25cyclictest7268-21chrt19:54:493
22522996627,25cyclictest7268-21chrt19:54:493
22514996649,7cyclictest0-21swapper/221:24:592
22514996647,13cyclictest25751-21/usr/sbin/munin23:40:172
22514996644,13cyclictest0-21swapper/220:14:532
2251499663,46cyclictest27693-21ssh21:54:532
22514996621,16cyclictest0-21swapper/219:25:222
22514996621,16cyclictest0-21swapper/219:25:222
22502996646,13cyclictest22488-21cyclictest19:50:271
22502996646,13cyclictest22488-21cyclictest19:50:271
2250299663,50cyclictest30724-21ssh21:05:111
2250299663,41cyclictest26533-21df_inode19:15:081
22522996532,23cyclictest14234-21chrt20:14:583
22522996532,22cyclictest23126-21chrt22:40:233
22522996532,22cyclictest23126-21chrt22:40:233
22522996532,21cyclictest31849-21chrt19:30:113
22522996532,21cyclictest17472-21chrt20:25:003
22522996531,21cyclictest23289-21chrt23:36:043
22522996531,21cyclictest23289-21chrt23:36:043
22522996531,21cyclictest14803-21chrt23:20:413
22522996531,20cyclictest6760-21chrt00:01:343
22522996531,20cyclictest6760-21chrt00:01:343
22522996530,22cyclictest4876-21taskset19:45:213
22522996529,25cyclictest3595-21taskset21:14:383
22522996529,25cyclictest3595-21taskset21:14:383
22514996547,12cyclictest18357-21unixbench_multi20:25:252
22514996543,15cyclictest22488-21cyclictest22:55:162
22502996546,13cyclictest7808-21sh22:15:161
22502996546,13cyclictest7808-21sh22:15:161
2250299653,46cyclictest1312-21/usr/sbin/munin22:05:111
2250299653,46cyclictest1312-21/usr/sbin/munin22:05:111
2250299653,43cyclictest724-21latency_hist19:34:521
22522996434,19cyclictest21823-21chrt20:37:033
22522996434,19cyclictest21823-21chrt20:37:033
22522996432,21cyclictest10505-21chrt20:04:493
22522996432,20cyclictest25053-21chrt21:50:103
22522996431,21cyclictest3877-21chrt19:42:463
22522996431,21cyclictest19408-21chrt22:35:003
22522996431,21cyclictest18487-21chrt23:27:463
22522996431,21cyclictest12526-21chrt20:10:083
22522996431,21cyclictest11809-21chrt23:15:343
22522996431,20cyclictest4703-21chrt23:05:003
22522996431,19cyclictest2396-21chrt23:55:053
22522996430,23cyclictest28928-21taskset23:45:083
22522996430,21cyclictest32356-21chrt22:01:413
22522996430,20cyclictest13208-21chrt21:30:113
22522996430,20cyclictest13208-21chrt21:30:113
22522996429,22cyclictest8615-21chrt19:56:193
22522996429,22cyclictest18041-21chrt21:38:143
22522996429,22cyclictest10193-21chrt21:25:093
2251499643,42cyclictest31879-21fschecks_count19:30:112
22502996449,8cyclictest0-21swapper/119:10:281
2250299644,35cyclictest8497-21switchtime23:10:241
2250299644,35cyclictest8497-21switchtime23:10:241
2250299643,41cyclictest22796-21latency22:40:141
2250299643,41cyclictest22796-21latency22:40:141
2250299643,40cyclictest4234-21latency_hist19:44:531
2250299643,39cyclictest8210-21/usr/sbin/munin00:05:211
2250299643,34cyclictest24345-21sed00:30:241
22502996423,32cyclictest29138-21cat23:45:131
22497996450,8cyclictest15350irq/295-2188000.ethernet19:56:020
22522996330,21cyclictest9771-21chrt00:06:413
22522996328,23cyclictest25940-21taskset22:45:223
2251499633,43cyclictest25625-21cstates20:50:062
22502996347,11cyclictest352-21runrttasks21:45:101
22502996341,12cyclictest0-21swapper/121:00:211
2250299633,42cyclictest352-21runrttasks19:37:251
2250299633,42cyclictest352-21runrttasks19:37:251
2193826312,33sleep30-21swapper/319:00:363
22522996231,19cyclictest29470-21chrt22:50:453
22522996231,19cyclictest29470-21chrt22:50:453
22522996230,21cyclictest1930-21chrt19:36:303
22522996230,21cyclictest1930-21chrt19:36:303
22522996230,19cyclictest29798-21chrt21:00:493
22522996229,21cyclictest25221-21chrt20:49:223
22522996229,21cyclictest25221-21chrt20:49:223
22522996229,20cyclictest23643-21chrt20:44:453
22522996229,20cyclictest23643-21chrt20:44:453
22522996229,20cyclictest11852-21chrt20:06:403
2250299627,36cyclictest23190-21latency_hist00:29:521
2250299627,36cyclictest23190-21latency_hist00:29:521
22502996244,12cyclictest352-21runrttasks19:45:001
22502996240,10cyclictest0-21swapper/121:12:201
22502996240,10cyclictest0-21swapper/121:12:201
22502996226,8cyclictest181rcu_preempt22:00:111
2250299622,41cyclictest20194-21ssh22:35:231
2250299612,36cyclictest26993-21latency_hist20:54:521
22522996031,19cyclictest25541-21chrt20:50:033
22514996038,12cyclictest0-21swapper/220:35:162
22514996038,12cyclictest0-21swapper/220:35:162
22522995929,20cyclictest25738-21chrt19:10:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional