You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 12:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sun May 03, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
153839911055,48cyclictest11979-21sh21:35:151
15373991043,77cyclictest15655-21apt-get19:02:520
15373991012,68cyclictest22495-21apt-get22:47:120
1392729926,40sleep10-21swapper/118:57:231
15395999710,68cyclictest0-21swapper/222:47:142
140522979,58sleep014092-21if_eth018:57:260
15395999642,46cyclictest0-21swapper/221:07:012
15395999642,46cyclictest0-21swapper/221:07:012
1539599963,36cyclictest31518-21fschecks_time22:07:212
15373999646,33cyclictest4129-21threads00:02:330
15373999646,33cyclictest4129-21threads00:02:330
1537399943,82cyclictest25235-21taskset21:57:130
1537399943,82cyclictest25235-21taskset21:57:130
15395999366,18cyclictest6481-21apt-key21:27:002
1539599933,35cyclictest22213-21ssh21:52:192
1539599933,29cyclictest27807-21latency_hist19:37:012
1539599922,36cyclictest30471-21ntp_kernel_pll_21:12:252
15373999263,21cyclictest1636-21latency_hist22:12:010
1539599914,29cyclictest24505-21/usr/sbin/munin23:42:332
15395999138,45cyclictest0-21swapper/223:32:332
1539599913,34cyclictest12293-21open_inodes23:22:292
1539599913,33cyclictest15646-21find19:02:082
1539599904,29cyclictest14179-21fschecks_time20:33:252
1539599904,29cyclictest14179-21fschecks_time20:33:252
1539599904,29cyclictest14179-21fschecks_time20:33:252
1539599903,35cyclictest19485-21apt22:41:572
15395999033,49cyclictest0-21swapper/221:17:162
1539599903,31cyclictest14043-21ssh21:37:362
15395999025,55cyclictest0-21swapper/221:27:082
1539599902,26cyclictest835-21/usr/sbin/munin23:57:262
15383999070,14cyclictest7061-21switchtime00:07:321
15373999067,15cyclictest1131-21ssh23:57:330
1537399903,71cyclictest5533-21apt-get20:07:220
1537399903,71cyclictest5533-21apt-get20:07:220
1539599894,33cyclictest29032-21/usr/sbin/munin22:57:322
15395998937,43cyclictest0-21swapper/222:32:232
1539599893,35cyclictest5966-21idleruntime-cro00:06:562
1539599893,35cyclictest5966-21idleruntime-cro00:06:562
1539599893,35cyclictest18596-21latency_hist20:47:002
1539599893,32cyclictest12622-21fschecks_time00:17:202
15395998930,50cyclictest0-21swapper/221:42:232
15395998867,15cyclictest11773-21latency_hist20:26:592
15395998865,15cyclictest14950-21fschecks_time23:27:202
1539599883,36cyclictest27281-21runrttasks21:07:532
1539599883,36cyclictest27281-21runrttasks21:07:532
1539599883,35cyclictest2283-21grep19:57:192
1539599883,34cyclictest978-21ntpq19:52:262
1539599883,31cyclictest19207-21ntpq20:47:252
1539599883,31cyclictest16369-21wc20:37:542
15395998820,19cyclictest10655-21readlink21:32:312
1539599874,31cyclictest6396-21cstates00:07:142
1539599873,34cyclictest19199-21fschecks_time21:47:192
1539599873,34cyclictest19199-21fschecks_time21:47:192
1539599873,32cyclictest17970-21latency_hist00:26:592
1539599873,27cyclictest25181-21/usr/sbin/munin21:57:572
1539599873,27cyclictest25181-21/usr/sbin/munin21:57:572
15395998726,8cyclictest0-21swapper/222:52:132
15395998720,27cyclictest18299-21sh00:27:082
15383998758,17cyclictest15687-21dpkg19:02:521
15373998738,32cyclictest0-21swapper/022:26:570
1539599864,33cyclictest28644-21latency22:02:232
15395998639,40cyclictest2822-21fschecks_count23:07:182
1539599863,34cyclictest352-21runrttasks19:17:282
1539599863,29cyclictest2810-21grep22:12:352
1539599863,29cyclictest22782-21latency_hist22:47:012
1539599863,28cyclictest9510-21wc20:17:292
15395998621,9cyclictest0-21swapper/223:21:582
15395998621,9cyclictest0-21swapper/223:21:582
1539599853,35cyclictest21681-21latency_hist20:56:592
1539599853,34cyclictest9937-21/usr/sbin/munin00:12:292
1539599853,34cyclictest5783-21grep22:17:332
1539599853,34cyclictest29343-21latency_hist19:42:012
1539599853,33cyclictest27033-21ssh23:47:212
1539599853,33cyclictest27033-21ssh23:47:212
1539599853,32cyclictest20662-21apt-get23:37:072
1539599853,31cyclictest352-21runrttasks19:22:222
1539599853,31cyclictest11248-21grep22:27:232
1539599853,30cyclictest7438-21tune2fs20:12:252
15395998527,49cyclictest0-21swapper/219:07:042
1539599843,35cyclictest7004-21latency_hist20:12:002
1539599843,35cyclictest7004-21latency_hist20:12:002
1539599843,34cyclictest29692-21/usr/sbin/munin23:52:212
1539599843,33cyclictest8835-21head22:22:342
1539599843,33cyclictest20114-21fschecks_count19:13:552
1539599843,32cyclictest352-21runrttasks20:29:332
1539599843,32cyclictest29702-21fschecks_time19:42:192
141122847,52sleep214119-21kernelversion18:57:282
1539599833,33cyclictest32349-21unixbench_singl19:48:522
1539599833,33cyclictest32349-21unixbench_singl19:48:522
1539599832,32cyclictest24865-21cut19:27:202
15395998314,22cyclictest2277-21cron23:06:572
15373998325,27cyclictest24101-21cat21:02:230
15373998325,27cyclictest24101-21cat21:02:230
1539599824,31cyclictest22203-21fschecks_time20:57:202
1539599823,34cyclictest5889-21ssh23:12:182
1539599823,31cyclictest5245-21runrttasks20:07:002
1539599823,31cyclictest5245-21runrttasks20:07:002
15383998262,15cyclictest30751-21unixbench_multi23:52:351
15383998262,14cyclictest6481-21apt-key21:27:001
15373998258,16cyclictest22821-21sh21:52:330
15373998256,19cyclictest352-21runrttasks23:52:080
15373998219,33cyclictest0-21swapper/019:57:000
15373998151,11cyclictest15-21ksoftirqd/022:31:560
1508228120,39sleep30-21swapper/319:01:583
15383998063,10cyclictest28176-21acpi22:02:071
15373998029,34cyclictest0-21swapper/023:22:240
15383997959,12cyclictest22196-21tune2fs21:52:191
1537399793,42cyclictest0-21swapper/022:02:130
1537399793,37cyclictest0-21swapper/022:12:250
15373997931,18cyclictest10401-21ssh21:32:230
15383997857,15cyclictest32034-21ssh22:07:341
15383997839,10cyclictest181rcu_preempt00:17:121
15383997834,35cyclictest25937-21modprobe19:31:571
15373997852,17cyclictest2374-21systemd-journal23:16:570
1537399784,49cyclictest6912-21apt-get21:27:180
15383997760,12cyclictest15350-21cyclictest21:48:481
15383997760,12cyclictest15350-21cyclictest21:48:481
15373997719,21cyclictest24174-21cat23:42:240
15383997658,13cyclictest15350-21cyclictest20:47:181
15383997645,7cyclictest181rcu_preempt19:57:311
1538399764,37cyclictest10277-21kworker/u19:1+ext4-rsv-conversion19:07:201
1538399763,64cyclictest22495-21apt-get22:47:191
15383997618,43cyclictest1049-21grep19:52:301
1537399763,38cyclictest31755-21munin-run23:01:570
15373997625,35cyclictest0-21swapper/021:23:420
1537399762,52cyclictest20643-21switchtime22:42:310
15383997557,11cyclictest24694-21kworker/u17:5-nfsiod23:31:061
15383997551,18cyclictest31022-21sh21:12:401
15373997547,19cyclictest14090-21cut22:32:180
1537399753,25cyclictest0-21swapper/021:51:380
1537399753,25cyclictest0-21swapper/021:51:380
1538399746,58cyclictest21956-21sshd20:57:121
15383997459,10cyclictest15350-21cyclictest19:37:011
15383997457,12cyclictest15350-21cyclictest22:16:341
15383997454,13cyclictest0-21swapper/121:47:001
15383997453,10cyclictest0-21swapper/123:11:581
15383997441,26cyclictest277-21dbus-daemon23:51:581
15383997441,26cyclictest277-21dbus-daemon23:51:581
1537399743,25cyclictest0-21swapper/021:46:070
15373997421,25cyclictest0-21swapper/022:52:420
15373997413,21cyclictest0-21swapper/021:18:290
15383997356,12cyclictest15350-21cyclictest23:22:341
15383997355,12cyclictest8446-21sh23:16:591
1538399733,54cyclictest22959-21sed22:47:021
1538399733,32cyclictest6103-21/usr/sbin/munin20:07:431
1538399733,32cyclictest6103-21/usr/sbin/munin20:07:431
1537399738,30cyclictest0-21swapper/020:27:230
1537399737,48cyclictest26855-21apt-get21:07:250
1537399737,48cyclictest26855-21apt-get21:07:250
1537399733,36cyclictest352-21runrttasks00:25:360
1537399732,38cyclictest352-21runrttasks19:48:470
1537399732,38cyclictest352-21runrttasks19:48:470
15383997255,11cyclictest297-21systemd-logind20:06:591
15383997255,11cyclictest297-21systemd-logind20:06:591
15383997254,12cyclictest20420-21ntp_kernel_pll_19:14:021
15383997249,15cyclictest0-21swapper/121:37:341
15383997249,13cyclictest0-21swapper/123:21:571
15383997249,13cyclictest0-21swapper/123:21:571
1538399723,55cyclictest24658-21apt-get21:57:201
1538399723,55cyclictest24658-21apt-get21:57:201
1537399727,54cyclictest16752-21kworker/u17:1+ext4-rsv-conversion19:07:210
15373997253,13cyclictest14674-21ssh23:27:130
15373997252,15cyclictest0-21swapper/000:28:290
15373997249,14cyclictest19676-21snmpd19:17:300
1537399723,43cyclictest29534-21ls23:52:000
1537399723,43cyclictest29534-21ls23:52:000
1537399723,32cyclictest20498-21latency_hist23:37:000
15383997149,12cyclictest0-21swapper/121:07:571
15383997149,12cyclictest0-21swapper/121:07:571
15383997143,20cyclictest1144-21ls23:57:331
15373997149,15cyclictest0-21swapper/023:07:330
15373997148,15cyclictest300-21rs:main0
15373997142,19cyclictest17148-21cpu20:42:140
1537399713,34cyclictest352-21runrttasks00:07:370
1537399713,27cyclictest0-21swapper/019:32:480
1537399712,51cyclictest19790-21apt-get19:12:300
1537399712,31cyclictest7661-21netstat20:12:310
15383997054,9cyclictest31790-21kworker/u17:1-ext4-rsv-conversion21:32:011
15383997050,14cyclictest277-21dbus-daemon19:42:001
15383997046,13cyclictest0-21swapper/123:47:001
15383997036,12cyclictest181rcu_preempt20:37:531
1538399702,48cyclictest13868-21apt-get20:32:211
1538399702,48cyclictest13868-21apt-get20:32:211
1538399702,48cyclictest13868-21apt-get20:32:211
15383997024,26cyclictest19750-21kworker/u17:0+xprtiod22:55:561
1537399703,32cyclictest11976-21munin-run00:16:570
1537399702,47cyclictest22918-21grep20:57:340
1537399702,34cyclictest6485-21ssh22:19:170
15383996944,16cyclictest0-21swapper/122:17:131
15383996940,20cyclictest10266-21sort00:12:341
1538399693,45cyclictest21859-21latency_hist20:57:011
1538399691,43cyclictest352-21runrttasks22:27:221
15373996951,12cyclictest297-21systemd-logind19:26:590
15373996942,19cyclictest12956-21cpu21:37:130
15373996917,34cyclictest13416-21rm00:17:370
15383996852,10cyclictest13397-21apt-key20:31:591
1538399684,47cyclictest8737-21ssh22:22:341
1538399684,43cyclictest3517-21latency_hist21:22:021
1538399684,42cyclictest352-21runrttasks19:21:131
1538399684,38cyclictest11153-21timerandwakeup20:22:331
15383996824,11cyclictest181rcu_preempt20:12:181
1537399682,37cyclictest24588-21/usr/sbin/munin19:27:220
15373996810,30cyclictest0-21swapper/022:38:360
1538399676,29cyclictest352-21runrttasks22:57:321
15383996753,10cyclictest23197-21kworker/u17:3-rpciod23:35:281
15383996752,9cyclictest20960-21kworker/u18:100:31:561
1538399674,48cyclictest14112-21fschecks_time22:32:181
1538399672,43cyclictest352-21runrttasks00:23:111
1537399673,32cyclictest2850-21cat19:57:330
1537399673,29cyclictest352-21runrttasks20:52:000
1537399673,24cyclictest0-21swapper/023:18:310
1537399673,24cyclictest0-21swapper/023:18:310
1538399664,43cyclictest8884-21cstates20:17:121
15383996640,15cyclictest0-21swapper/100:02:311
15383996640,15cyclictest0-21swapper/100:02:311
1538399663,40cyclictest314-21latency_hist19:52:011
1538399663,40cyclictest314-21latency_hist19:52:011
15373996646,14cyclictest2387-21latency_hist23:06:590
15373996641,16cyclictest15350-21cyclictest20:52:090
15373996640,17cyclictest30793-21latency_hist19:47:010
1537399663,32cyclictest3966-21fschecks_time20:02:190
1537399663,32cyclictest3966-21fschecks_time20:02:190
1537399663,29cyclictest0-21swapper/020:33:310
1537399663,29cyclictest0-21swapper/020:33:310
1537399663,29cyclictest0-21swapper/020:33:310
1537399663,25cyclictest0-21swapper/021:13:390
15383996542,17cyclictest26507-21latency_hist21:07:001
15383996542,17cyclictest26507-21latency_hist21:07:001
15383996540,12cyclictest0-21swapper/123:38:031
1538399652,42cyclictest23700-21switchtime19:22:321
1538399652,24cyclictest32249-21ssh23:02:161
15373996527,30cyclictest0-21swapper/019:42:010
15402996411,39cyclictest269742sleep321:07:283
15402996411,39cyclictest269742sleep321:07:283
1538399643,44cyclictest30078-21mailstats19:42:301
1537399643,40cyclictest16823-21modprobe20:41:570
15383996330,10cyclictest181rcu_preempt20:47:001
15373996330,23cyclictest0-21swapper/023:37:320
1540299627,39cyclictest24678-21chrt22:49:063
1540299623,40cyclictest24353-21chrt23:42:303
15383996227,12cyclictest181rcu_preempt22:37:181
1537399623,35cyclictest10376-21/usr/sbin/munin20:22:230
1540299613,35cyclictest917-21kworker/3:1+events19:42:193
1540299594,37cyclictest5245-21taskset20:07:003
1540299594,37cyclictest5245-21taskset20:07:003
1540299593,39cyclictest530-21taskset19:52:143
1540299593,36cyclictest3225-21taskset21:21:593
1540299592,39cyclictest30024-21chrt22:57:333
1540299583,39cyclictest17073-21chrt20:42:073
1540299582,38cyclictest917-21kworker/3:1+events21:54:443
1540299578,28cyclictest0-21swapper/323:52:003
1540299578,28cyclictest0-21swapper/323:52:003
1540299573,40cyclictest24153-21taskset21:02:253
1540299573,40cyclictest24153-21taskset21:02:253
1540299573,38cyclictest25039-21chrt19:27:233
1540299573,36cyclictest3327-21taskset23:07:293
1540299572,35cyclictest917-21kworker/3:1+events19:22:303
1540299563,36cyclictest12190-21chrt23:22:253
1540299554,34cyclictest22546-21taskset22:46:583
1540299553,40cyclictest28056-21taskset22:02:013
1540299553,40cyclictest28056-21taskset22:02:013
1540299553,38cyclictest26340-21chrt22:52:183
1540299553,32cyclictest22102-21chrt19:17:393
1540299547,26cyclictest0-21swapper/320:32:173
1540299547,26cyclictest0-21swapper/320:32:173
1540299547,26cyclictest0-21swapper/320:32:173
1540299544,32cyclictest19063-21chrt20:47:223
1540299543,36cyclictest15778-21chrt23:27:393
1540299543,35cyclictest14306-21taskset21:38:213
1540299543,34cyclictest2536-21chrt23:07:013
1540299543,34cyclictest16915-21chrt22:37:143
1540299543,33cyclictest927-21chrt23:57:303
1540299542,38cyclictest3691-21taskset21:22:113
1540299542,38cyclictest14172-21taskset22:32:203
1540299542,36cyclictest7918-21chrt22:22:153
1540299542,34cyclictest10434-21chrt23:19:293
1540299542,34cyclictest10434-21chrt23:19:293
1540299533,37cyclictest19584-21chrt21:47:293
1540299533,37cyclictest19584-21chrt21:47:293
1540299533,34cyclictest11597-21chrt20:25:233
1540299532,35cyclictest30391-21chrt21:12:223
1540299532,34cyclictest19997-21chrt19:13:143
1540299524,36cyclictest18096-21taskset23:32:213
1540299523,36cyclictest2693-21chrt22:12:343
1540299523,36cyclictest10208-21chrt00:12:333
1540299523,35cyclictest1711-21chrt22:12:013
1540299522,38cyclictest31676-21chrt19:48:373
1540299522,38cyclictest31676-21chrt19:48:373
1540299522,36cyclictest16305-21chrt21:42:223
1540299522,35cyclictest6991-21chrt21:27:223
1540299522,33cyclictest20555-21chrt20:52:203
15402995211,27cyclictest0-21swapper/319:07:203
1540299513,35cyclictest11169-21chrt22:27:213
1540299513,33cyclictest19750-21chrt00:28:343
1540299513,32cyclictest18144-21chrt00:27:003
1540299512,36cyclictest8324-21chrt00:10:143
291472500,9sleep30-21swapper/322:02:343
1540299505,28cyclictest8058-21chrt23:16:123
1540299504,32cyclictest4107-21chrt00:02:323
1540299504,32cyclictest4107-21chrt00:02:323
1540299503,36cyclictest16132-21chrt19:03:483
15402995031,9cyclictest0-21swapper/320:27:183
1540299502,35cyclictest23381-21chrt21:02:003
1540299502,27cyclictest0-21swapper/320:07:173
1540299502,27cyclictest0-21swapper/320:07:173
1540299493,34cyclictest9511-21chrt20:17:293
1540299478,27cyclictest12600-21taskset00:17:203
1540299473,30cyclictest16373-21chrt20:37:543
1540299464,29cyclictest2353-21chrt19:57:203
1540299462,31cyclictest7511-21chrt20:12:273
1540299453,30cyclictest11339-21chrt21:33:263
1540299443,30cyclictest5937-21chrt22:17:453
1540299442,30cyclictest26249-21taskset19:32:133
1540299432,29cyclictest28194-21taskset19:37:243
1540299422,30cyclictest29691-21chrt23:52:073
1540299422,27cyclictest20666-21taskset23:37:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional