You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 11:47
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sun Apr 12, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
279809913072,13cyclictest21863-21/usr/sbin/munin23:23:410
280339910150,44cyclictest11863-21sh21:18:002
280339910143,52cyclictest7601-21kernelversion22:58:482
280339910142,52cyclictest2339-21timerwakeupswit20:59:222
280339910036,9cyclictest0-21swapper/220:38:562
27888210015,55sleep127901-21apt-config19:03:261
27888210015,55sleep127901-21apt-config19:03:261
28033999933,12cyclictest0-21swapper/223:23:242
28033999842,9cyclictest0-21swapper/222:24:002
28033999839,12cyclictest0-21swapper/221:13:222
28033999839,12cyclictest0-21swapper/221:13:222
28033999839,12cyclictest0-21swapper/221:13:222
28033999634,13cyclictest0-21swapper/221:18:392
2803399962,36cyclictest15742-21latency_hist20:03:242
28033999444,12cyclictest1-21systemd21:43:562
28033999444,12cyclictest1-21systemd21:43:562
2803399943,38cyclictest13439-21ntp_states22:13:512
2803399943,34cyclictest20932-21fschecks_count20:18:442
28033999431,12cyclictest0-21swapper/223:33:582
28010999471,13cyclictest0-21swapper/120:13:421
28033999344,8cyclictest0-21swapper/200:19:002
28010999372,11cyclictest0-21swapper/120:48:541
28010999370,13cyclictest0-21swapper/122:03:481
161182930,6sleep30-21swapper/320:03:453
28033999134,17cyclictest18388-21cstates21:28:392
28010999171,12cyclictest0-21swapper/119:08:251
28010999170,11cyclictest0-21swapper/119:33:251
28033999024,20cyclictest470-21cut21:53:242
2774929017,47sleep227761-21munin-run19:03:232
2774929017,47sleep227761-21munin-run19:03:232
2803399893,35cyclictest19418-21idleruntime20:13:432
275902896,63sleep00-21swapper/019:01:250
275902896,63sleep00-21swapper/019:01:250
28010998873,8cyclictest0-21swapper/121:33:381
28010998873,8cyclictest0-21swapper/121:33:381
28010998870,9cyclictest0-21swapper/120:48:241
28010998771,8cyclictest0-21swapper/119:43:241
28010998753,13cyclictest181rcu_preempt23:31:231
27980998755,7cyclictest181rcu_preempt23:08:180
27980998755,7cyclictest181rcu_preempt23:08:180
28033998633,10cyclictest0-21swapper/200:14:372
28010998670,7cyclictest0-21swapper/119:09:011
28010998670,7cyclictest0-21swapper/119:09:011
28010998669,8cyclictest0-21swapper/119:58:581
28010998648,8cyclictest181rcu_preempt20:34:001
2778828610,45sleep30-21swapper/319:03:243
2778828610,45sleep30-21swapper/319:03:243
28033998546,10cyclictest27968-21cyclictest19:19:352
28033998538,9cyclictest0-21swapper/222:38:402
2803399853,43cyclictest21941-21apt-get22:28:372
2803399853,43cyclictest21941-21apt-get22:28:372
28010998471,7cyclictest0-21swapper/120:03:491
28010998468,7cyclictest0-21swapper/122:29:541
28010998468,7cyclictest0-21swapper/122:29:541
28010998453,24cyclictest6854-40logrotate23:53:241
28010998445,28cyclictest7401-21kworker/u18:0+nfsiod22:21:221
27980998462,16cyclictest15736-21latency_hist20:03:240
28033998254,18cyclictest803-21sendmail-mta23:40:192
2803399823,35cyclictest24907-21ssh22:33:512
28010998241,9cyclictest181rcu_preempt21:18:381
28033998130,15cyclictest13608-21ntp_kernel_pll_23:08:522
28033998119,30cyclictest16582-21switchtime22:18:552
28010998168,7cyclictest0-21swapper/120:23:451
28010998168,7cyclictest0-21swapper/120:23:451
28010998140,7cyclictest181rcu_preempt20:23:241
27980998154,17cyclictest24096-21latency_hist21:38:250
27980998154,17cyclictest24096-21latency_hist21:38:250
28033998057,14cyclictest300-21rs:main2
2803399803,33cyclictest24514-21fschecks_time21:38:452
27980998061,14cyclictest7786-21sh22:58:510
27980998060,14cyclictest4943-21sh22:53:560
27980998018,26cyclictest15-21ksoftirqd/021:05:180
28010997963,11cyclictest27968-21cyclictest22:38:471
28010997958,14cyclictest28343-21netstat23:33:491
28010997958,14cyclictest27968-21cyclictest20:08:511
28010997954,8cyclictest181rcu_preempt19:48:411
27980997957,14cyclictest1382-21idleruntime21:54:030
27980997946,12cyclictest9720-21kworker/u17:2-ext4-rsv-conversion23:18:240
28033997860,11cyclictest32172-21kworker/u17:1-rpciod23:53:402
2803399783,60cyclictest27755-21apt-get19:03:382
2803399783,31cyclictest2795-21diskmemload23:30:162
28033997754,17cyclictest30600-21sendmail00:33:212
28033997754,17cyclictest30600-21sendmail00:33:212
28033997747,22cyclictest544-21cron19:28:222
28033997747,22cyclictest544-21cron19:28:222
28033997735,13cyclictest181rcu_preempt23:13:322
28033997630,29cyclictest2795-21diskmemload22:58:212
28010997659,10cyclictest1162-21apt-key22:48:251
28010997659,10cyclictest1162-21apt-key22:48:251
27980997662,9cyclictest13902-21/usr/sbin/munin00:03:550
27980997661,9cyclictest23132-21unixbench_multi22:30:270
27980997661,9cyclictest23132-21unixbench_multi22:30:270
27980997656,14cyclictest30369-21cut22:43:230
28033997553,16cyclictest31127-21switchtime20:48:572
28010997555,14cyclictest544-21cron23:53:261
28010997548,9cyclictest31-21ksoftirqd/123:12:001
28010997544,9cyclictest181rcu_preempt22:38:101
28010997541,8cyclictest181rcu_preempt21:04:291
27980997555,14cyclictest18871-21apt-key00:13:230
27980997538,9cyclictest181rcu_preempt20:28:370
28033997445,21cyclictest17872-21latency20:08:482
28010997460,8cyclictest0-21swapper/121:53:241
28010997458,11cyclictest27968-21cyclictest00:28:451
28010997458,11cyclictest27968-21cyclictest00:28:451
28010997443,8cyclictest181rcu_preempt21:08:401
28010997443,8cyclictest181rcu_preempt21:08:401
28010997443,8cyclictest181rcu_preempt21:08:401
27980997457,10cyclictest9039-21apt-key21:13:250
27980997457,10cyclictest9039-21apt-key21:13:250
27980997457,10cyclictest9039-21apt-key21:13:250
27980997446,20cyclictest18419-21modprobe22:23:210
2803399733,44cyclictest6919-21cat23:53:242
28010997359,7cyclictest0-21swapper/120:58:241
28010997358,9cyclictest13250-21ssh22:13:471
28010997358,9cyclictest13199-21cpuspeed00:03:371
28010997354,12cyclictest277-21dbus-daemon21:03:241
28010997344,9cyclictest181rcu_preempt00:00:471
27980997341,12cyclictest2706-21sh19:23:260
27980997341,12cyclictest2706-21sh19:23:260
27980997340,11cyclictest27968-21cyclictest20:28:230
27980997340,11cyclictest27968-21cyclictest20:28:230
28033997246,17cyclictest4755-21fschecks_count19:28:422
28010997253,13cyclictest7561-21modprobe19:38:211
28010997251,15cyclictest20979-21sh21:33:241
28010997237,8cyclictest181rcu_preempt23:23:281
28010997235,7cyclictest0-21swapper/122:49:401
28010997235,7cyclictest0-21swapper/122:49:401
2801099723,25cyclictest10766-21wc23:03:541
2801099723,25cyclictest10766-21wc23:03:541
28010997230,11cyclictest0-21swapper/119:28:121
28010997230,11cyclictest0-21swapper/119:28:121
27980997244,21cyclictest15506-21kworker/u17:1+nfsiod00:21:580
27980997239,8cyclictest181rcu_preempt21:43:360
27980997239,8cyclictest181rcu_preempt21:43:360
28033997155,11cyclictest300-21rs:main2
28033997155,11cyclictest300-21rs:main2
28033997153,11cyclictest13758-21/usr/sbin/munin00:03:522
28033997126,35cyclictest19009-21gpgv00:13:252
28010997156,11cyclictest27968-21cyclictest23:18:241
28010997154,10cyclictest21470-21apt-key22:28:231
28010997152,14cyclictest2795-21diskmemload22:59:411
28010997151,11cyclictest352-21runrttasks22:09:021
28010997144,20cyclictest24208-21fschecks_time20:28:451
27980997155,12cyclictest27968-21cyclictest23:43:330
27980997155,10cyclictest15500-21systemd-run22:18:220
27980997134,10cyclictest58-21kcompactd023:58:570
2803399705,36cyclictest0-21swapper/219:18:252
28033997042,22cyclictest10084-21fschecks_count22:08:442
28033997033,8cyclictest181rcu_preempt22:06:492
2803399702,36cyclictest3728-21ntp_offset21:03:522
28010997055,9cyclictest31516-21mailstats23:38:551
28010997055,8cyclictest0-21swapper/121:55:421
28010997041,10cyclictest181rcu_preempt22:54:371
28010997038,9cyclictest181rcu_preempt00:17:291
27980997051,13cyclictest20549-21latency_hist20:18:240
27980997043,21cyclictest2898-21ssh21:03:250
27980997041,20cyclictest29208-21grep20:43:470
27980997039,11cyclictest3050-21sh22:50:480
27980997039,11cyclictest3050-21sh22:50:480
27980997034,8cyclictest181rcu_preempt20:08:370
28033996954,10cyclictest58-21kcompactd021:33:492
28033996954,10cyclictest58-21kcompactd021:33:492
28033996950,13cyclictest1402-21ssh21:54:042
28033996937,23cyclictest0-21swapper/220:33:212
28010996954,8cyclictest0-21swapper/123:44:151
28010996952,12cyclictest18967-21apt-key00:13:241
28010996949,11cyclictest0-21swapper/119:53:431
28010996949,11cyclictest0-21swapper/119:53:431
28010996945,11cyclictest0-21swapper/120:43:231
28010996943,11cyclictest181rcu_preempt23:20:461
27980996953,11cyclictest27968-21cyclictest22:06:380
27980996943,20cyclictest24968-21kworker/u17:2+nfsiod21:15:240
27980996941,21cyclictest22118-21kworker/u17:4+nfsiod22:35:120
27980996924,23cyclictest181rcu_preempt21:52:250
28033996850,12cyclictest298-21in:imuxsock21:28:212
28033996850,12cyclictest298-21in:imuxsock21:28:212
28033996826,33cyclictest1382-21gpgv20:58:252
2803399682,36cyclictest24587-21latency_hist23:28:242
28010996847,9cyclictest0-21swapper/119:19:541
27980996854,9cyclictest9720-21kworker/u17:2+xprtiod23:45:510
27980996854,10cyclictest27968-21cyclictest21:20:060
27980996852,10cyclictest7401-21kworker/u18:0-nfsiod21:31:300
27980996852,10cyclictest29110-21kworker/u18:0-rpciod00:13:290
27980996851,12cyclictest1-21systemd22:48:230
27980996851,12cyclictest1-21systemd22:48:230
27980996842,9cyclictest181rcu_preempt19:13:350
27980996837,22cyclictest9838-21cat19:43:480
27980996825,28cyclictest18921-21/usr/sbin/munin23:18:500
27980996818,23cyclictest18740-21acpi22:23:340
28033996748,12cyclictest1491-21cpuspeed23:44:082
2803399673,29cyclictest6578-21munin-run22:03:222
2803399673,29cyclictest6578-21munin-run22:03:222
28033996725,19cyclictest31286-21unixbench_singl19:09:012
28033996725,19cyclictest31286-21unixbench_singl19:09:012
28010996753,7cyclictest0-21swapper/121:43:511
28010996753,7cyclictest0-21swapper/121:43:511
28010996748,13cyclictest2952-21systemd-journal22:00:301
28010996748,13cyclictest2952-21systemd-journal22:00:301
28010996730,7cyclictest181rcu_preempt19:43:481
27980996754,9cyclictest20925-21kworker/u17:0+rpciod23:33:510
27980996752,10cyclictest27968-21cyclictest23:30:570
27980996751,11cyclictest929-21apt-get20:58:230
27980996740,21cyclictest26484-21kworker/u17:2+nfsiod00:26:330
27980996739,22cyclictest3900-21/usr/sbin/munin21:58:430
27980996739,22cyclictest3900-21/usr/sbin/munin21:58:430
27980996737,10cyclictest0-21swapper/019:53:450
27980996737,10cyclictest0-21swapper/019:53:450
27980996722,35cyclictest30604-21idleruntime-cro00:33:210
27980996722,35cyclictest30604-21idleruntime-cro00:33:210
28033996647,14cyclictest12803-21munin-plugin-st00:03:232
28033996640,17cyclictest13505-21timerwakeupswit19:53:572
28033996640,17cyclictest13505-21timerwakeupswit19:53:572
2803399662,35cyclictest6622-21kworker/2:2+events_freezable_pwr_efficient22:49:342
2803399662,35cyclictest6622-21kworker/2:2+events_freezable_pwr_efficient22:49:342
28010996651,9cyclictest24118-21ssh00:21:351
28010996647,8cyclictest0-21swapper/121:23:551
28010996647,8cyclictest0-21swapper/121:23:551
28010996647,10cyclictest0-21swapper/121:38:501
28010996646,7cyclictest181rcu_preempt21:15:111
27980996650,10cyclictest10449-21ntp_offset22:08:510
27980996639,21cyclictest14753-21kworker/u17:0+nfsiod23:12:540
27980996639,20cyclictest32172-21kworker/u17:1+nfsiod23:57:580
27980996638,21cyclictest0-21swapper/023:49:360
2803399658,27cyclictest0-21swapper/219:43:542
2803399653,41cyclictest29010-21fschecks_count20:43:432
2803399652,30cyclictest31001-21fschecks_time22:43:452
2803399652,30cyclictest31001-21fschecks_time22:43:452
28010996550,11cyclictest27968-21cyclictest00:26:201
27980996546,13cyclictest7749-21latency_hist19:38:250
27980996540,19cyclictest2475-21kworker/u17:0+nfsiod21:26:160
27980996540,19cyclictest2475-21kworker/u17:0+nfsiod21:26:160
28033996426,8cyclictest181rcu_preempt20:33:572
28033996415,9cyclictest6842-21mailstats19:33:542
27980996449,8cyclictest0-21swapper/019:53:230
2803399633,30cyclictest0-21swapper/200:24:082
28033996310,39cyclictest8074-21cut19:38:442
28010996341,16cyclictest866-21latency_hist19:18:251
28033996249,7cyclictest0-21swapper/223:03:592
28033996249,7cyclictest0-21swapper/223:03:592
2798099613,40cyclictest22237-21latency_hist20:23:250
2798099613,30cyclictest25141-21sort21:39:000
27980996128,22cyclictest29634-21modprobe19:08:220
2798099612,32cyclictest2835-21sort19:23:250
2804799603,31cyclictest917-21kworker/3:1+events20:33:513
2803399602,26cyclictest16026-21taskset20:03:402
27980996046,7cyclictest0-21swapper/020:03:520
27980996044,10cyclictest4768-21fschecks_count19:28:420
27980996043,11cyclictest26941-21systemd-run20:38:220
27980996029,21cyclictest8516-21cat19:38:560
2798099594,32cyclictest31913-21latency_hist19:13:260
2798099594,32cyclictest31913-21latency_hist19:13:260
27980995823,10cyclictest352-21runrttasks20:48:480
2798099572,31cyclictest27665-21ntp_kernel_err20:38:510
2804799552,33cyclictest917-21kworker/3:1+events20:08:483
28047995436,11cyclictest135502sleep300:03:473
28047995336,10cyclictest300812sleep323:37:233
2804799522,33cyclictest3665-21chrt23:47:483
2804799522,33cyclictest19306-21chrt00:13:353
28047995129,13cyclictest451rcuc/323:03:543
28047995129,13cyclictest451rcuc/323:03:543
2804799512,30cyclictest917-21kworker/3:1+events23:28:403
2804799503,33cyclictest1414-21taskset22:48:353
2804799503,33cyclictest1414-21taskset22:48:353
2804799495,32cyclictest3128-21chrt21:03:353
2804799492,31cyclictest20538-21taskset22:25:583
28047994832,9cyclictest171622chrt22:19:463
28047994817,21cyclictest22514-21chrt23:23:533
2804799473,35cyclictest28080-21chrt21:43:583
2804799473,35cyclictest28080-21chrt21:43:583
2804799473,33cyclictest2768-21chrt21:56:023
2804799473,27cyclictest13592-21chrt21:19:293
2804799463,31cyclictest29474-21taskset22:40:493
2804799463,31cyclictest17731-21taskset23:15:503
2804799463,30cyclictest20135-21taskset23:19:143
2804799463,27cyclictest917-21kworker/3:1+events22:33:323
2804799462,32cyclictest15703-21taskset21:23:473
2804799462,32cyclictest15703-21taskset21:23:473
2804799462,28cyclictest23156-21chrt22:30:273
2804799462,28cyclictest23156-21chrt22:30:273
2804799454,27cyclictest10121-21taskset23:58:313
2804799453,30cyclictest30855-21taskset19:08:533
2804799453,30cyclictest30855-21taskset19:08:533
2804799453,30cyclictest21429-21taskset21:33:403
2804799453,30cyclictest21429-21taskset21:33:403
2804799453,30cyclictest10090-21chrt21:14:513
2804799453,29cyclictest12333-21taskset22:12:563
2804799453,28cyclictest7348-21taskset23:53:403
2804799452,30cyclictest30747-21taskset23:38:323
2804799443,31cyclictest32257-21taskset22:45:233
2804799443,31cyclictest32257-21taskset22:45:233
2804799443,31cyclictest19637-21taskset21:29:473
2804799443,31cyclictest15401-21taskset22:18:113
2804799443,30cyclictest4811-21chrt23:48:543
2804799443,29cyclictest7197-21chrt22:58:353
2804799443,29cyclictest6218-21taskset19:33:363
2804799443,29cyclictest23543-21taskset20:26:463
2804799443,29cyclictest23543-21taskset20:26:463
2804799443,29cyclictest20742-21taskset20:18:363
2804799443,28cyclictest4288-21taskset22:53:363
2804799443,26cyclictest7539-21chrt21:09:563
2804799443,26cyclictest7539-21chrt21:09:563
2804799443,26cyclictest7539-21chrt21:09:563
2804799442,30cyclictest30426-21taskset21:48:443
2804799442,29cyclictest26337-21chrt00:24:323
2804799442,28cyclictest917-21kworker/3:1+events22:03:083
2804799442,28cyclictest917-21kworker/3:1+events22:03:083
2804799433,30cyclictest4585-21taskset19:28:343
2804799433,28cyclictest9572-21taskset19:43:403
2804799433,28cyclictest19134-21taskset20:13:333
2804799433,27cyclictest30411-21taskset20:48:373
2804799433,27cyclictest24182-21taskset21:38:303
2804799433,27cyclictest1444-21taskset20:58:373
2804799433,27cyclictest12670-21chrt19:53:373
2804799433,27cyclictest12670-21chrt19:53:373
28047994327,11cyclictest305492sleep300:33:213
28047994327,11cyclictest305492sleep300:33:213
2804799432,30cyclictest27190-21chrt20:38:363
2804799432,30cyclictest14925-21chrt19:58:553
2804799432,29cyclictest22163-21chrt00:18:363
2804799432,29cyclictest11041-21chrt19:48:383
2804799432,28cyclictest13001-21taskset23:08:323
2804799423,29cyclictest8444-21taskset22:05:183
2804799423,28cyclictest18044-21taskset00:11:283
2804799423,28cyclictest1346-21taskset19:18:293
2804799423,27cyclictest31988-21chrt19:13:353
2804799423,27cyclictest2982-21chrt19:23:353
2804799423,27cyclictest2982-21chrt19:23:353
2804799423,27cyclictest28885-21chrt20:43:393
2804799423,27cyclictest28313-21taskset19:03:353
2804799413,27cyclictest7796-21chrt19:38:303
2804799413,27cyclictest32016-21taskset20:53:313
2804799392,24cyclictest24165-21chrt20:28:443
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional