You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 06:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Wed May 27, 2026 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2214622610,6sleep3451rcuc/318:58:383
224519911941,40cyclictest0-21swapper/200:20:202
224519911941,40cyclictest0-21swapper/200:20:202
22451991103,34cyclictest2102-21latency_hist22:55:222
21131210911,69sleep221195-21fschecks_time18:55:492
22451991033,32cyclictest9834-21sh21:20:462
22451991033,32cyclictest9834-21sh21:20:462
224519910317,49cyclictest0-21swapper/200:00:472
224519910317,49cyclictest0-21swapper/200:00:472
22451991023,86cyclictest14596-21ssh22:21:592
22451991023,86cyclictest14596-21ssh22:21:592
22451991002,32cyclictest11876-21wc23:10:452
224339910066,27cyclictest4279-21apt-key22:05:240
22433999826,59cyclictest18817-21kworker/u19:0+ext4-rsv-conversion19:20:430
2245199973,81cyclictest0-21swapper/223:20:352
22451999671,17cyclictest277-21dbus-daemon23:35:212
22451999666,22cyclictest25893-21unixbench_singl20:45:582
2245199963,70cyclictest4406-21apt-get22:05:472
22451999620,42cyclictest0-21swapper/221:05:372
2245199953,78cyclictest0-21swapper/220:50:372
2245199953,78cyclictest0-21swapper/220:50:372
22433999564,22cyclictest15350irq/295-2188000.ethernet22:05:380
22451999440,39cyclictest0-21swapper/200:10:312
2243399933,68cyclictest10703-21ssh22:15:370
2243399933,68cyclictest10703-21ssh22:15:370
22451999249,36cyclictest18756-21fschecks_time20:25:442
2243399922,68cyclictest352-21runrttasks00:15:350
2243399922,68cyclictest352-21runrttasks00:15:350
2245199915,73cyclictest0-21swapper/219:45:352
22443999154,13cyclictest181rcu_preempt19:00:331
2243399919,50cyclictest15-21ksoftirqd/019:02:490
2245199902,47cyclictest25178-21ntp_states19:05:522
2243399903,34cyclictest15-21ksoftirqd/023:45:220
22451998966,16cyclictest277-21dbus-daemon19:55:222
2245199893,38cyclictest11704-21systemctl22:16:462
2245199893,38cyclictest11704-21systemctl22:16:462
2245199892,20cyclictest6400-21/usr/sbin/munin21:15:532
22443998955,15cyclictest181rcu_preempt22:35:331
22443998955,15cyclictest181rcu_preempt22:35:331
22443998942,11cyclictest181rcu_preempt20:30:441
2244399892,78cyclictest13347-21apt-get22:20:351
2244399892,78cyclictest13347-21apt-get22:20:351
2243399897,47cyclictest18103-21apt-get21:36:380
2243399893,50cyclictest15-21ksoftirqd/022:12:150
22451998872,10cyclictest12609-21kworker/u17:0-xprtiod21:36:532
22451998869,12cyclictest10158-21apt-config22:15:212
22433998820,40cyclictest15-21ksoftirqd/021:44:530
2245199874,75cyclictest0-21swapper/220:35:362
22451998745,35cyclictest0-21swapper/221:30:232
2245199872,35cyclictest6966-21/usr/sbin/munin23:56:352
22443998768,14cyclictest22422-21cyclictest21:30:231
22433998759,18cyclictest2585-21latency_hist23:50:240
22433998759,18cyclictest2585-21latency_hist23:50:240
2243399874,69cyclictest8822-21apt-get19:55:360
22451998643,35cyclictest0-21swapper/222:30:412
2245199862,28cyclictest352-21runrttasks19:11:022
22433998651,26cyclictest22473-21ssh22:35:070
22451998538,39cyclictest0-21swapper/221:35:222
2245199853,67cyclictest0-21swapper/220:55:372
2245199853,66cyclictest0-21swapper/219:15:312
2245199852,6cyclictest0-21swapper/223:35:342
2245199852,6cyclictest0-21swapper/223:35:342
2245199852,33cyclictest9034-21ntp_states23:05:512
22443998567,12cyclictest297-21systemd-logind22:30:211
2244399852,62cyclictest10979-21taskset22:16:241
2244399852,62cyclictest10979-21taskset22:16:241
2243399853,46cyclictest15-21ksoftirqd/020:47:460
2243399853,45cyclictest15-21ksoftirqd/022:51:210
22433998515,43cyclictest15-21ksoftirqd/020:42:330
2245199842,35cyclictest2644-21/usr/sbin/munin22:55:422
22443998443,13cyclictest181rcu_preempt19:35:201
22443998443,13cyclictest181rcu_preempt19:35:201
22443998436,11cyclictest181rcu_preempt23:50:421
2244399843,47cyclictest19267-21fschecks_time21:36:391
2244399843,36cyclictest28550-21munin-run22:45:201
22433998425,37cyclictest15-21ksoftirqd/019:50:280
22451998334,38cyclictest0-21swapper/223:30:242
2245199833,36cyclictest15348-21expr23:15:562
2245199833,33cyclictest516-21apt-get19:30:322
2245199833,33cyclictest516-21apt-get19:30:322
2245199832,7cyclictest0-21swapper/220:10:352
2245199832,7cyclictest0-21swapper/220:10:352
2245199832,33cyclictest16991-21/usr/sbin/munin20:20:392
22443998352,13cyclictest181rcu_preempt00:20:381
22443998348,13cyclictest25280-21ssh21:45:531
2244399833,32cyclictest24201-21munin-node19:05:281
22433998344,31cyclictest0-21swapper/021:15:580
22433998336,27cyclictest15-21ksoftirqd/023:50:450
2243399833,51cyclictest15-21ksoftirqd/020:51:270
2243399833,51cyclictest15-21ksoftirqd/020:51:270
2155328313,36sleep10-21swapper/118:56:041
22451998238,36cyclictest0-21swapper/223:45:492
22451998238,36cyclictest0-21swapper/223:45:492
22451998238,36cyclictest0-21swapper/221:10:592
22451998238,36cyclictest0-21swapper/221:10:592
2245199823,32cyclictest25117-21/usr/sbin/munin21:45:472
2245199822,32cyclictest29947-21grep21:00:372
2245199822,32cyclictest29947-21grep21:00:372
22443998250,10cyclictest181rcu_preempt20:45:481
22433998257,16cyclictest297-21systemd-logind20:25:210
2243399823,48cyclictest15-21ksoftirqd/000:20:550
22433998211,56cyclictest15-21ksoftirqd/023:04:020
22433998211,56cyclictest15-21ksoftirqd/023:04:020
2245199813,7cyclictest0-21swapper/219:20:502
2245199813,66cyclictest7314-21ssh23:03:102
2245199813,66cyclictest7314-21ssh23:03:102
2245199813,33cyclictest23406-21/usr/sbin/munin20:40:382
2245199812,35cyclictest21322-21/usr/sbin/munin00:20:432
2245199812,31cyclictest18011-21taskset22:26:012
2244399814,43cyclictest29732-21apt-get19:20:381
22433998152,20cyclictest297-21systemd-logind21:35:210
22433998115,47cyclictest15-21ksoftirqd/020:40:240
22433998115,37cyclictest15-21ksoftirqd/022:39:240
22433998115,37cyclictest15-21ksoftirqd/022:39:240
2245199804,61cyclictest3735-21apt-get19:40:332
2245199804,61cyclictest3735-21apt-get19:40:332
22451998037,35cyclictest0-21swapper/222:40:232
22451998037,35cyclictest0-21swapper/222:40:232
22451998033,39cyclictest0-21swapper/222:00:342
2245199803,32cyclictest23520-21rm21:43:062
2245199803,30cyclictest20575-21/usr/sbin/munin20:30:462
2245199802,35cyclictest12038-21/usr/sbin/munin20:05:312
2245199802,35cyclictest12038-21/usr/sbin/munin20:05:312
2245199802,33cyclictest32079-21sh21:56:512
2245199802,33cyclictest14160-21taskset00:08:562
2245199802,33cyclictest14160-21taskset00:08:562
2245199802,31cyclictest30902-21sh23:42:072
2245199802,30cyclictest2473-21/usr/sbin/munin19:35:432
2245199802,30cyclictest2473-21/usr/sbin/munin19:35:432
22443998055,19cyclictest6094-21threads19:45:541
2244399802,56cyclictest4464-21sh22:05:401
22433998057,16cyclictest15084-21ssh23:15:530
2243399803,47cyclictest15-21ksoftirqd/000:10:290
2243399803,37cyclictest15-21ksoftirqd/021:58:250
2243399802,53cyclictest15-21ksoftirqd/023:32:190
2243399802,39cyclictest15-21ksoftirqd/021:12:180
2243399802,39cyclictest15-21ksoftirqd/021:12:180
22433998015,38cyclictest15-21ksoftirqd/021:29:080
22433998012,38cyclictest15-21ksoftirqd/022:25:410
2245199793,69cyclictest0-21swapper/219:55:472
2245199793,32cyclictest31957-21/usr/sbin/munin19:25:522
2245199793,29cyclictest22781-21seq19:00:432
2245199793,28cyclictest4381-21seq23:51:552
2245199792,34cyclictest15399-21expr20:15:382
2245199792,34cyclictest15399-21expr20:15:382
2245199792,31cyclictest31263-21taskset22:49:372
2245199792,31cyclictest26678-21systemctl00:30:212
2245199792,31cyclictest26267-21if_eth022:40:452
22443997962,11cyclictest352-21runrttasks19:40:531
22443997962,11cyclictest352-21runrttasks19:40:531
22443997960,14cyclictest22422-21cyclictest22:57:381
22443997948,15cyclictest20763-21ssh22:30:551
2244399793,35cyclictest24217-21/usr/sbin/munin00:25:421
2243399799,58cyclictest13885-21apt-get22:20:350
2243399799,58cyclictest13885-21apt-get22:20:350
22433997953,18cyclictest26204-21fschecks_time22:40:440
2243399794,43cyclictest15-21ksoftirqd/021:50:220
2243399794,40cyclictest15-21ksoftirqd/019:20:210
22433997940,14cyclictest29549-21latency_hist21:00:230
2243399793,43cyclictest15-21ksoftirqd/023:24:490
22433997926,30cyclictest15-21ksoftirqd/021:05:450
22433997926,30cyclictest15-21ksoftirqd/020:01:470
2243399792,45cyclictest15-21ksoftirqd/023:36:160
2243399792,45cyclictest15-21ksoftirqd/023:36:160
2245199783,32cyclictest27790-21cat21:50:382
2245199782,30cyclictest11221-21/usr/sbin/munin20:00:572
22443997858,12cyclictest26577-21/usr/sbin/munin20:50:321
22443997858,12cyclictest26577-21/usr/sbin/munin20:50:321
22443997847,12cyclictest181rcu_preempt19:10:311
22443997837,10cyclictest181rcu_preempt23:30:151
22443997835,11cyclictest181rcu_preempt23:19:401
22443997829,24cyclictest0-21swapper/100:10:361
22433997860,11cyclictest22422-21cyclictest00:02:050
22433997860,11cyclictest22422-21cyclictest00:02:050
2243399783,45cyclictest15-21ksoftirqd/023:11:580
2243399783,44cyclictest19884-21modprobe20:30:210
2243399783,42cyclictest15-21ksoftirqd/020:19:000
2243399783,42cyclictest15-21ksoftirqd/020:19:000
2243399783,37cyclictest15-21ksoftirqd/020:35:070
2243399783,30cyclictest15-21ksoftirqd/019:14:420
2243399782,42cyclictest15-21ksoftirqd/022:46:580
22433997810,45cyclictest15-21ksoftirqd/023:05:320
22443997759,13cyclictest22422-21cyclictest21:15:421
22443997756,9cyclictest181rcu_preempt00:16:441
22443997756,9cyclictest181rcu_preempt00:16:441
22443997745,23cyclictest848-21sh21:05:481
22443997744,13cyclictest181rcu_preempt21:23:131
22443997744,13cyclictest181rcu_preempt21:23:131
2243399779,31cyclictest15-21ksoftirqd/019:40:210
2243399779,31cyclictest15-21ksoftirqd/019:40:210
22433997758,13cyclictest2862-21kworker/u17:3+rpciod23:56:180
2243399773,52cyclictest15-21ksoftirqd/022:55:490
2243399773,34cyclictest15-21ksoftirqd/021:50:440
22433997722,31cyclictest15-21ksoftirqd/019:30:360
22433997722,31cyclictest15-21ksoftirqd/019:30:360
22443997654,15cyclictest29553-21diskmemload23:11:571
22443997646,13cyclictest181rcu_preempt21:03:381
22443997646,13cyclictest181rcu_preempt21:03:381
22443997644,11cyclictest181rcu_preempt20:05:251
22443997644,11cyclictest181rcu_preempt20:05:251
22443997630,11cyclictest181rcu_preempt23:35:441
22443997630,11cyclictest181rcu_preempt23:35:441
22443997617,34cyclictest0-21swapper/123:45:391
22443997617,34cyclictest0-21swapper/123:45:391
22433997630,20cyclictest15-21ksoftirqd/020:12:200
22433997630,20cyclictest15-21ksoftirqd/020:12:200
22433997620,33cyclictest15-21ksoftirqd/000:10:200
22433997620,33cyclictest15-21ksoftirqd/000:10:200
2233027612,40sleep022346-21munin-run19:00:230
22443997549,8cyclictest181rcu_preempt20:00:451
22443997542,14cyclictest181rcu_preempt23:35:071
22443997536,12cyclictest3851-21modprobe22:05:201
22443997523,12cyclictest0-21swapper/119:15:361
2243399753,38cyclictest352-21runrttasks21:21:550
2243399753,38cyclictest352-21runrttasks21:21:550
2243399752,35cyclictest15-21ksoftirqd/023:28:450
2243399752,35cyclictest15-21ksoftirqd/020:08:510
2243399752,35cyclictest15-21ksoftirqd/020:08:510
22433997510,48cyclictest15-21ksoftirqd/021:00:360
22433997510,48cyclictest15-21ksoftirqd/021:00:360
22443997451,16cyclictest352-21runrttasks20:35:571
22443997443,12cyclictest181rcu_preempt22:14:551
22443997433,16cyclictest181rcu_preempt23:09:131
22443997425,22cyclictest0-21swapper/123:03:101
22443997425,22cyclictest0-21swapper/123:03:101
22433997411,37cyclictest15-21ksoftirqd/000:25:440
22443997357,9cyclictest0-21swapper/122:50:241
22443997345,9cyclictest181rcu_preempt23:45:231
22443997341,12cyclictest181rcu_preempt21:53:201
2244399733,19cyclictest0-21swapper/120:41:011
2244399732,28cyclictest8587-21latency_hist19:55:241
22443997245,9cyclictest181rcu_preempt19:55:511
22443997236,10cyclictest181rcu_preempt19:25:541
2243399723,28cyclictest15-21ksoftirqd/019:40:430
2243399723,28cyclictest15-21ksoftirqd/019:40:430
22433997211,35cyclictest15-21ksoftirqd/019:46:190
22443997140,12cyclictest181rcu_preempt21:40:351
2244399712,43cyclictest10141-21users00:00:591
2244399712,43cyclictest10141-21users00:00:591
22443997043,10cyclictest181rcu_preempt21:11:031
22443997043,10cyclictest181rcu_preempt21:11:031
22443997040,12cyclictest181rcu_preempt23:57:571
22443997033,7cyclictest181rcu_preempt19:35:431
22443997033,7cyclictest181rcu_preempt19:35:431
22443997033,12cyclictest181rcu_preempt21:31:331
22443997028,13cyclictest181rcu_preempt23:22:151
22443997024,9cyclictest181rcu_preempt20:25:431
2244399702,46cyclictest29549-21latency_hist21:00:231
22433997049,8cyclictest181rcu_preempt19:05:270
2243399704,39cyclictest15-21ksoftirqd/019:25:450
22443996953,10cyclictest660-21ssh22:51:201
22443996945,13cyclictest0-21swapper/120:25:231
22443996937,8cyclictest181rcu_preempt20:16:041
22443996937,8cyclictest181rcu_preempt20:16:041
22443996848,8cyclictest181rcu_preempt00:05:381
22443996848,8cyclictest181rcu_preempt00:05:381
22443996637,7cyclictest181rcu_preempt21:55:291
22443996539,15cyclictest0-21swapper/120:15:231
22443996539,15cyclictest0-21swapper/120:15:231
2245699634,45cyclictest2590-21taskset22:01:473
2245699607,33cyclictest6209-21chrt23:55:393
22456995833,19cyclictest0-21swapper/319:20:383
2245699578,31cyclictest0-21swapper/322:05:363
2245699573,39cyclictest8854-21taskset19:55:503
2245699563,39cyclictest9526-21chrt22:13:423
2245699563,36cyclictest13585-21chrt20:10:243
2245699563,36cyclictest13585-21chrt20:10:243
2245699543,37cyclictest11624-21chrt23:10:403
2245699543,36cyclictest25910-21chrt22:40:373
2245699542,40cyclictest30844-21chrt21:55:393
2245699534,35cyclictest8570-21chrt23:05:373
2245699533,38cyclictest27963-21taskset21:50:443
2245699533,38cyclictest16273-21taskset20:15:593
2245699533,38cyclictest16273-21taskset20:15:593
2245699533,37cyclictest26699-21chrt19:10:463
2245699533,34cyclictest3245-21chrt21:10:233
22456995316,24cyclictest0-21swapper/322:15:403
22456995316,24cyclictest0-21swapper/322:15:403
2245699523,39cyclictest809-21chrt22:51:443
22456995217,22cyclictest0-21swapper/322:20:353
22456995217,22cyclictest0-21swapper/322:20:353
2245699513,37cyclictest25049-21chrt21:45:463
2245699513,37cyclictest18714-21chrt20:25:433
2245699513,35cyclictest29496-21chrt22:45:523
2245699513,34cyclictest17534-21chrt00:15:033
2245699512,36cyclictest8723-21taskset21:19:193
2245699512,36cyclictest5599-21chrt19:45:423
2245699512,36cyclictest32513-21chrt21:05:163
2245699512,36cyclictest32513-21chrt21:05:163
22456995123,17cyclictest0-21swapper/300:15:353
22456995123,17cyclictest0-21swapper/300:15:353
2245699506,34cyclictest19547-21chrt22:30:203
2245699504,33cyclictest24320-21taskset19:05:283
2245699503,37cyclictest4938-21chrt21:12:053
2245699503,37cyclictest4938-21chrt21:12:053
2245699503,33cyclictest3966-21chrt22:57:113
2245699503,30cyclictest9358-21chrt21:20:243
2245699503,30cyclictest9358-21chrt21:20:243
2245699502,38cyclictest29010-21taskset20:55:563
2245699502,34cyclictest22626-21chrt22:35:223
2245699493,35cyclictest18733-21chrt21:35:443
2245699493,35cyclictest17563-21taskset23:20:373
2245699493,35cyclictest14669-21chrt21:29:123
2245699493,34cyclictest21523-21taskset00:20:473
2245699493,33cyclictest9355-21chrt00:00:443
2245699493,33cyclictest9355-21chrt00:00:443
2245699493,33cyclictest11552-21chrt20:02:393
2245699484,33cyclictest1310-21chrt19:30:553
2245699484,33cyclictest1310-21chrt19:30:553
2245699483,36cyclictest32006-21taskset19:25:533
2245699483,35cyclictest32604-21chrt23:45:423
2245699483,35cyclictest32604-21chrt23:45:423
2245699483,34cyclictest24818-21chrt20:45:233
2245699483,33cyclictest2584-21chrt19:35:453
2245699483,33cyclictest2584-21chrt19:35:453
2245699483,31cyclictest20782-21chrt20:30:543
2245699474,32cyclictest26847-21chrt00:30:233
2245699473,32cyclictest28159-21chrt20:55:243
2245699472,36cyclictest16620-21chrt23:19:033
2245699472,33cyclictest27946-21chrt19:15:363
2245699463,34cyclictest8359-21taskset19:55:213
2245699463,34cyclictest6760-21chrt23:01:363
2245699463,34cyclictest6760-21chrt23:01:363
2245699463,33cyclictest31227-21chrt23:43:023
2245699463,33cyclictest21808-21chrt21:40:393
2245699463,33cyclictest17214-21chrt20:20:453
2245699463,33cyclictest16047-21chrt21:30:503
2245699463,32cyclictest24094-21chrt19:05:243
2245699463,31cyclictest23190-21chrt22:35:433
2245699463,31cyclictest23190-21chrt22:35:433
2245699462,32cyclictest22679-21taskset23:29:163
2245699462,31cyclictest27785-21chrt23:36:463
2245699462,31cyclictest27785-21chrt23:36:463
2245699462,31cyclictest12434-21chrt00:05:473
2245699462,31cyclictest12434-21chrt00:05:473
2245699462,29cyclictest15086-21chrt20:15:223
2245699462,29cyclictest15086-21chrt20:15:223
2245699454,31cyclictest4486-21chrt19:40:563
2245699454,31cyclictest4486-21chrt19:40:563
2245699453,32cyclictest5198-21chrt23:54:133
2245699453,32cyclictest23714-21taskset23:30:423
2245699442,32cyclictest25710-21chrt20:45:563
2245699433,31cyclictest22533-21chrt20:35:563
2245699393,26cyclictest26573-21taskset20:50:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional