You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-18 - 08:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Wed Mar 18, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2665821240,7sleep2371rcuc/222:40:292
1297621180,6sleep30-21swapper/322:17:573
266569911030,63cyclictest29559-21apt-get23:40:160
26671991047,87cyclictest32283-21apt-get22:51:062
266569910444,28cyclictest6521-21ssh23:00:330
266569910444,28cyclictest6521-21ssh23:00:330
26671991022,68cyclictest30110-21apt-get23:40:162
26671991013,68cyclictest18950-21apt-get00:15:162
26671991013,68cyclictest18950-21apt-get00:15:162
26678991002,58cyclictest0-21swapper/300:15:173
26678991002,58cyclictest0-21swapper/300:15:173
26656999938,34cyclictest16126-21seq23:16:010
26656999739,30cyclictest16041-21seq22:23:090
26656999739,30cyclictest16041-21seq22:23:090
26656999631,31cyclictest0-21swapper/021:10:150
2666799952,27cyclictest8841-21seq19:47:261
2666799943,56cyclictest32195-21apt-get21:00:131
2666799943,56cyclictest32195-21apt-get21:00:131
26656999431,38cyclictest0-21swapper/022:02:500
2667199933,58cyclictest22172-21apt-get00:20:162
26656999343,42cyclictest28688-21awk20:50:010
26656999342,43cyclictest21412-21/usr/sbin/munin23:25:290
26656999339,21cyclictest19266-21seq22:28:480
26656999334,24cyclictest31756-21seq19:19:030
26656999332,35cyclictest16628-21ssh21:30:210
26656999327,27cyclictest29981-21runrttasks20:51:140
2666799923,50cyclictest26669-21sed19:05:061
26656999257,23cyclictest22467-21ssh21:40:160
26656999242,43cyclictest19791-21ssh23:22:550
26656999242,43cyclictest19791-21ssh23:22:550
26656999241,21cyclictest13734-21unixbench_multi00:05:380
26656999233,29cyclictest352-21runrttasks22:16:480
26656999229,41cyclictest0-21swapper/022:41:080
26656999168,15cyclictest200-21systemd-journal21:07:160
26656999140,41cyclictest22172-21apt-get00:20:280
26656999139,23cyclictest9763-21latency19:51:180
26656999139,23cyclictest9763-21latency19:51:180
26656999136,41cyclictest0-21swapper/020:05:150
26656999135,29cyclictest1209-21ssh23:45:280
26656999135,29cyclictest1209-21ssh23:45:280
26656999132,34cyclictest0-21swapper/022:07:200
2666799903,47cyclictest13904-21latency_hist20:05:051
2666799903,42cyclictest23866-21ssh23:30:141
2666799903,42cyclictest23866-21ssh23:30:141
26656999070,14cyclictest25397-21fschecks_count00:25:250
26656999041,39cyclictest5153-21/usr/sbin/munin19:35:340
26656999015,34cyclictest0-21swapper/020:58:400
26656998954,24cyclictest18950-21apt-get00:15:290
26656998954,24cyclictest18950-21apt-get00:15:290
26656998940,39cyclictest31187-21ssh21:55:020
26656998940,21cyclictest25757-21modprobe22:40:010
26656998926,31cyclictest4505-21runrttasks22:57:090
2656628915,48sleep226578-21ls19:05:052
26671998862,16cyclictest10767-21latency_hist19:55:042
26671998862,16cyclictest10767-21latency_hist19:55:042
2666799884,41cyclictest7526-21ssh23:55:371
2666799884,41cyclictest7526-21ssh23:55:371
26656998845,35cyclictest26645-21cyclictest19:05:050
26656998840,38cyclictest8228-21ssh21:15:350
26656998838,41cyclictest1851-21rm22:53:110
26656998838,39cyclictest11306-21ssh23:09:240
26671998767,12cyclictest15523-21ssh00:10:042
2666799873,39cyclictest933-21ssh23:45:221
2666799873,39cyclictest933-21ssh23:45:221
26656998740,40cyclictest200-21systemd-journal20:15:010
26656998740,37cyclictest29275-21awk22:45:210
26656998738,39cyclictest3651-21sshd23:50:140
2666799863,39cyclictest12111-21fschecks_count23:10:241
2666799863,39cyclictest12111-21fschecks_count23:10:241
2666799863,27cyclictest352-21runrttasks19:18:011
2666799862,33cyclictest6406-21date23:55:031
26656998652,21cyclictest25437-21apt-get21:45:120
26656998640,24cyclictest0-21swapper/023:15:050
26656998640,24cyclictest0-21swapper/023:15:050
26656998636,40cyclictest9574-21ssh22:11:550
2666799855,75cyclictest0-21swapper/123:20:151
2666799855,75cyclictest0-21swapper/123:20:151
26667998543,35cyclictest6712-21processes19:40:321
26667998541,36cyclictest0-21swapper/123:10:011
2666799853,7cyclictest0-21swapper/120:50:171
2666799853,41cyclictest28903-21runrttasks00:30:351
2666799853,38cyclictest15708-21sshd00:10:151
2666799853,38cyclictest15708-21sshd00:10:151
2666799853,36cyclictest2841-21/usr/sbin/munin22:55:401
2666799852,37cyclictest352-21runrttasks00:21:131
26656998560,14cyclictest26831-21apt-get19:05:120
26656998560,14cyclictest26831-21apt-get19:05:120
26656998557,22cyclictest23858-21sh23:30:130
26656998557,22cyclictest23858-21sh23:30:130
26656998554,20cyclictest28154-21apt-get00:30:170
26656998549,30cyclictest0-21swapper/021:35:120
26656998547,33cyclictest0-21swapper/023:55:130
26656998547,33cyclictest0-21swapper/023:55:130
26656998541,35cyclictest27317-21/usr/sbin/munin23:35:260
26671998455,20cyclictest14573-21ssh23:15:022
26671998455,20cyclictest14573-21ssh23:15:022
2666799844,69cyclictest0-21swapper/122:40:131
2666799843,42cyclictest15844-21cpuspeed20:10:181
2666799843,40cyclictest31171-21idleruntime-cro21:55:011
2666799843,37cyclictest11834-21ssh21:21:281
2666799843,33cyclictest31752-21apt-get22:50:281
2666799843,30cyclictest27422-21/usr/sbin/munin23:35:281
2666799842,40cyclictest19676-21snmpd20:20:211
26656998465,11cyclictest22247-21apt-get20:30:180
26656998440,35cyclictest25634-21taskset20:40:190
2666799834,38cyclictest2833-21/usr/sbin/munin22:00:371
26667998342,33cyclictest0-21swapper/100:05:271
26667998340,36cyclictest0-21swapper/121:50:041
2666799833,8cyclictest0-21swapper/120:30:181
2666799833,33cyclictest27835-21latency_hist00:30:041
2666799833,33cyclictest23548-21modprobe23:30:011
2666799833,32cyclictest5551-21/usr/sbin/munin22:05:301
2666799833,32cyclictest24323-21/usr/sbin/munin20:35:251
2666799833,31cyclictest26812-21/usr/sbin/munin19:05:281
2666799833,31cyclictest26812-21/usr/sbin/munin19:05:281
2666799832,38cyclictest20154-21expr22:30:171
2666799832,32cyclictest1854-21/usr/sbin/munin21:05:271
26667998317,58cyclictest277-21dbus-daemon22:20:031
26656998344,28cyclictest0-21swapper/021:25:130
26656998340,35cyclictest20320-21fschecks_time22:30:220
2667899823,59cyclictest0-21swapper/322:51:063
26671998267,10cyclictest29815-21sed19:10:412
2666799824,68cyclictest0-21swapper/120:05:131
2666799824,37cyclictest32046-21latency_hist22:50:051
2666799823,40cyclictest9477-21taskset19:50:211
2666799823,40cyclictest9477-21taskset19:50:211
2666799823,39cyclictest21848-21cmp00:20:031
2666799823,39cyclictest21848-21cmp00:20:031
2666799823,38cyclictest14981-21taskset21:26:541
2666799823,34cyclictest1907-21grep19:25:331
2666799823,33cyclictest4630-21fschecks_count21:10:201
2666799823,33cyclictest308-21modprobe23:45:001
2666799823,33cyclictest29838-21grep19:10:411
2666799822,35cyclictest15809-21taskset22:22:301
2666799822,35cyclictest15809-21taskset22:22:301
2666799822,32cyclictest11635-21/usr/sbin/munin19:55:351
26656998229,20cyclictest0-21swapper/020:04:590
2587428222,37sleep30-21swapper/319:00:463
2667899813,46cyclictest0-21swapper/300:20:163
26667998158,11cyclictest5876-21systemd-run19:40:031
2666799815,66cyclictest0-21swapper/120:55:141
2666799814,9cyclictest0-21swapper/121:40:171
2666799813,40cyclictest6029-21/usr/sbin/munin23:00:191
2666799813,40cyclictest6029-21/usr/sbin/munin23:00:191
2666799813,37cyclictest352-21runrttasks21:30:161
2666799813,34cyclictest9883-21/usr/sbin/munin00:00:191
2666799813,33cyclictest8287-21fschecks_time22:10:221
2666799813,31cyclictest4249-21sendmail-msp19:35:021
2666799813,30cyclictest17601-21/usr/sbin/munin22:25:301
2666799812,40cyclictest32107-21ssh21:55:341
2666799812,40cyclictest32107-21ssh21:55:341
2666799812,33cyclictest21520-21/usr/sbin/munin20:25:371
26656998120,24cyclictest0-21swapper/000:14:220
26656998120,24cyclictest0-21swapper/000:14:220
2666799803,37cyclictest19851-21ssh21:35:251
2666799803,35cyclictest9696-21sh21:18:521
2666799803,32cyclictest25959-21/usr/sbin/munin20:40:281
2666799803,31cyclictest32587-21/usr/sbin/munin19:20:291
2666799803,30cyclictest17593-21/usr/sbin/munin20:15:221
2666799802,31cyclictest14894-21apt-get23:15:131
2666799802,30cyclictest23466-21/usr/sbin/munin22:35:261
26656998051,19cyclictest20640-21apt-get20:25:130
26656998042,30cyclictest10367-21ps00:00:330
2666799792,31cyclictest27532-21chrt20:45:451
26656997958,11cyclictest10952-21apt-get19:55:170
26656997949,23cyclictest174042sleep020:15:120
26656997939,33cyclictest32109-21rm21:55:340
26656997939,33cyclictest32109-21rm21:55:340
26656997939,33cyclictest10706-21fschecks_time21:20:220
26656997924,25cyclictest0-21swapper/019:20:360
26671997862,12cyclictest26645-21cyclictest19:40:362
26656997841,27cyclictest3625-21awk19:30:350
26671997759,11cyclictest0-21swapper/222:20:382
26671997759,11cyclictest0-21swapper/222:20:382
26671997750,17cyclictest13730-21latency_hist22:20:052
26656997720,21cyclictest0-21swapper/021:00:250
26656997720,21cyclictest0-21swapper/021:00:250
26671997650,19cyclictest1-21systemd19:59:162
26656997644,22cyclictest24527-21/usr/sbin/munin20:35:320
26656997644,21cyclictest28636-21munin-node19:10:100
26671997554,14cyclictest297-21systemd-logind20:55:022
2667199754,43cyclictest39-21ksoftirqd/220:41:062
26656997541,22cyclictest6089-21/usr/sbin/munin19:40:390
26678997416,45cyclictest0-21swapper/323:40:203
26671997457,11cyclictest300-21rs:main2
2667199742,50cyclictest352-21runrttasks21:41:032
26656997442,24cyclictest19207-21fschecks_count20:20:240
26656997343,20cyclictest9268-21latency_hist19:50:100
26671997230,20cyclictest0-21swapper/221:20:242
2667199722,36cyclictest32048-21latency_hist21:00:042
26656997237,24cyclictest1272-21sshd19:25:150
26671997151,14cyclictest300-21rs:main2
26671997053,11cyclictest23810-21kworker/u17:3-rpciod23:55:022
26671997047,14cyclictest7907-21cut19:45:232
26671997043,18cyclictest1520-21cut19:25:222
2667199703,47cyclictest10735-21tail00:00:372
2667199703,40cyclictest19365-21if_eth020:20:262
26671997022,28cyclictest39-21ksoftirqd/221:49:132
26671997019,30cyclictest39-21ksoftirqd/220:19:092
2667199697,35cyclictest39-21ksoftirqd/220:39:322
26671996946,13cyclictest0-21swapper/200:30:042
26671996935,11cyclictest181rcu_preempt22:12:582
2667199692,38cyclictest39-21ksoftirqd/219:36:412
26671996849,12cyclictest0-21swapper/223:35:012
26671996849,12cyclictest0-21swapper/223:35:012
26671996846,7cyclictest181rcu_preempt23:00:192
26671996846,7cyclictest181rcu_preempt23:00:192
26671996813,38cyclictest14779-21uname20:05:342
26671996749,13cyclictest13470-21cpu21:25:182
26671996749,11cyclictest0-21swapper/221:55:432
26671996749,11cyclictest0-21swapper/221:55:432
26671996747,11cyclictest6618-21kworker/u17:2-xprtiod00:10:362
26671996747,11cyclictest6618-21kworker/u17:2-xprtiod00:10:362
26671996740,9cyclictest181rcu_preempt23:36:492
26671996740,16cyclictest0-21swapper/222:30:372
2667199673,48cyclictest32201-21latency_hist22:50:062
2667199673,46cyclictest352-21runrttasks23:27:192
26671996733,10cyclictest181rcu_preempt21:39:002
26671996730,27cyclictest7605-21timerwakeupswit23:55:382
26671996730,27cyclictest7605-21timerwakeupswit23:55:382
26671996722,36cyclictest4283-21ls19:35:042
26671996713,37cyclictest17243-21wc20:15:042
26671996713,19cyclictest381ktimers/222:02:412
26671996644,12cyclictest0-21swapper/219:20:042
26671996634,9cyclictest181rcu_preempt23:24:022
26671996634,9cyclictest181rcu_preempt23:24:022
2667199663,30cyclictest39-21ksoftirqd/220:02:032
2667199663,27cyclictest23900-21grep22:35:362
26671996629,18cyclictest39-21ksoftirqd/221:09:092
26671996616,25cyclictest1086-21diskmemload22:05:352
2667199657,26cyclictest39-21ksoftirqd/221:17:502
26671996546,12cyclictest0-21swapper/220:30:042
26671996546,10cyclictest0-21swapper/223:16:152
26671996545,10cyclictest0-21swapper/220:30:342
26671996544,15cyclictest1419-21sh23:45:342
26671996544,15cyclictest1419-21sh23:45:342
26671996541,18cyclictest526-21runrttasks21:00:372
26671996541,18cyclictest526-21runrttasks21:00:372
2667199653,36cyclictest39-21ksoftirqd/221:33:362
26671996532,8cyclictest181rcu_preempt00:33:082
26671996445,11cyclictest0-21swapper/222:59:152
26671996432,9cyclictest181rcu_preempt21:54:192
26671996344,7cyclictest181rcu_preempt23:09:312
26671996311,20cyclictest381ktimers/221:13:402
2667199612,26cyclictest39-21ksoftirqd/219:05:472
2667199612,26cyclictest39-21ksoftirqd/219:05:472
2667899603,44cyclictest3556-21taskset23:50:053
2667899603,44cyclictest3556-21taskset23:50:053
2667199592,38cyclictest32313-21fschecks_count19:20:232
2667899553,38cyclictest27252-21chrt20:45:063
2667899543,37cyclictest990-21chrt19:25:023
2667899543,37cyclictest4882-21taskset23:51:023
2667899543,34cyclictest21089-21chrt22:30:373
2667899542,40cyclictest16227-21chrt20:10:283
2667899542,39cyclictest4340-21chrt21:10:053
2667899542,35cyclictest30472-21taskset19:15:053
2667899534,36cyclictest5206-21taskset22:59:063
2667899534,36cyclictest24734-21taskset20:35:343
26678995339,9cyclictest451rcuc/319:37:223
2667899533,39cyclictest24666-21chrt22:37:023
2667899533,36cyclictest19108-21chrt21:35:013
2667899532,42cyclictest8801-21taskset21:16:213
2667899532,38cyclictest9142-21chrt23:59:213
2667899532,38cyclictest9142-21chrt23:59:213
26678995310,25cyclictest0-21swapper/320:45:173
26678995239,9cyclictest451rcuc/322:27:593
2667899523,37cyclictest17959-21chrt20:15:303
2667899523,36cyclictest1721-21chrt22:00:023
2667899523,36cyclictest1721-21chrt22:00:023
2667899523,34cyclictest31552-21chrt20:55:393
2667899522,36cyclictest30151-21chrt21:52:203
2667899522,36cyclictest29715-21taskset20:50:373
2667899522,36cyclictest23119-21chrt23:28:533
2667899522,33cyclictest9306-21chrt19:50:123
2667899522,33cyclictest9306-21chrt19:50:123
2667899514,33cyclictest26986-21chrt23:35:173
2667899513,39cyclictest25373-21chrt00:25:233
2667899513,35cyclictest4864-21chrt22:05:043
2667899512,38cyclictest19175-21chrt20:20:203
2667899512,37cyclictest23826-21chrt23:30:113
2667899512,37cyclictest23826-21chrt23:30:113
2667899512,36cyclictest30151-21taskset00:33:093
2667899503,35cyclictest13624-21chrt21:25:213
2667899502,39cyclictest27956-21taskset22:42:493
2667899493,33cyclictest3345-21taskset19:30:283
2667899493,32cyclictest30582-21taskset19:15:183
2667899483,36cyclictest17452-21taskset23:19:453
2667899482,34cyclictest13643-21chrt00:05:373
2667899482,31cyclictest20914-21chrt21:36:393
2667899473,35cyclictest8346-21chrt22:10:233
2667899473,35cyclictest13433-21chrt20:00:373
2667899473,32cyclictest5611-21chrt21:10:543
2667899473,32cyclictest23604-21chrt20:35:023
2667899473,31cyclictest11451-21taskset23:09:483
2667899473,30cyclictest13293-21chrt23:11:353
2667899473,30cyclictest13293-21chrt23:11:353
2667899473,30cyclictest11578-21chrt19:55:343
2667899473,29cyclictest24074-21taskset21:42:063
2667899472,33cyclictest31664-21chrt22:49:503
2667899464,28cyclictest526-21chrt21:00:373
2667899464,28cyclictest526-21chrt21:00:373
2667899463,32cyclictest17999-21chrt00:14:003
2667899463,32cyclictest17999-21chrt00:14:003
2667899463,30cyclictest26213-21chrt21:45:353
2667899453,33cyclictest7461-21chrt23:02:023
2667899453,33cyclictest7461-21chrt23:02:023
2667899452,29cyclictest21282-21chrt20:25:343
2667899443,28cyclictest11269-21chrt21:20:353
2667899442,30cyclictest6458-21chrt22:06:343
2667899442,30cyclictest12142-21chrt00:04:243
2667899423,31cyclictest7817-21taskset19:45:183
2667899423,29cyclictest2646-21chrt19:30:043
2667899413,27cyclictest19194-21chrt23:21:153
2667899413,27cyclictest19194-21chrt23:21:153
26678994129,8cyclictest451rcuc/319:07:093
26678994129,8cyclictest451rcuc/319:07:093
2667899412,31cyclictest14912-21taskset22:20:363
2667899412,31cyclictest14912-21taskset22:20:363
2667899403,27cyclictest7078-21chrt19:40:373
2667899394,26cyclictest14796-21chrt20:05:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional