You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 13:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Thu Apr 30, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19907211316,60sleep219932-21munin-run19:02:102
20061991045,81cyclictest27108-21apt-get23:02:271
200569910435,31cyclictest19815-21ssh23:52:230
20066991033,43cyclictest7054-21ssh21:22:212
200569910125,32cyclictest7040-21apt-get21:22:200
20056999911,43cyclictest16840-21chrt23:45:330
20056999872,16cyclictest30268-21latency_hist19:32:120
20056999826,26cyclictest0-21swapper/023:32:200
20056999814,48cyclictest352-21runrttasks22:41:070
199812978,64sleep1277-21dbus-daemon19:02:131
2005699963,69cyclictest19501-21ssh23:52:090
20056999634,42cyclictest0-21swapper/022:10:180
20061999567,19cyclictest352-21runrttasks22:12:101
20066999434,50cyclictest0-21swapper/220:22:112
20066999344,41cyclictest0-21swapper/222:32:382
20066999344,41cyclictest0-21swapper/222:32:382
20061999372,13cyclictest9422-21apt23:32:091
20061999372,13cyclictest9422-21apt23:32:091
20056999370,17cyclictest23769-21/usr/sbin/munin21:52:110
2006699923,34cyclictest17036-21fschecks_count22:42:302
2006699922,36cyclictest25121-21fschecks_time00:02:302
2006699922,33cyclictest30153-21latency_hist19:32:112
20061999252,12cyclictest181rcu_preempt00:14:261
20066999139,44cyclictest0-21swapper/221:03:182
20066999139,44cyclictest0-21swapper/221:03:182
2006699913,36cyclictest17906-21latency20:32:362
2006699913,34cyclictest5301-21ntp_kernel_pll_23:22:382
2006699913,33cyclictest29658-21ssh00:12:112
2006699913,33cyclictest29658-21ssh00:12:112
20056999119,32cyclictest15610-21cron20:27:090
20056999119,32cyclictest15610-21cron20:27:090
2006699903,33cyclictest23008-21grep19:07:462
2006699903,33cyclictest16511-21find20:27:442
2006699903,31cyclictest11042-21grep21:27:462
2006699893,33cyclictest12350-21latency_hist20:17:132
2006699893,32cyclictest14320-21taskset23:41:512
2006699884,28cyclictest22532-21latency_hist23:57:102
20066998840,40cyclictest0-21swapper/223:32:102
20066998840,40cyclictest0-21swapper/223:32:102
2006699883,34cyclictest23808-21latency_hist20:52:122
2006699883,33cyclictest5206-21timerwakeupswit19:52:452
2006699883,33cyclictest5206-21timerwakeupswit19:52:452
2006699883,32cyclictest20487-21cut19:02:312
2006699882,34cyclictest21642-21unixbench_multi20:42:462
2006699882,34cyclictest20070-21unixbench_multi21:42:472
2006699882,34cyclictest20070-21unixbench_multi21:42:472
2006699874,29cyclictest31870-21latency_hist19:37:122
2006699873,36cyclictest4517-21latency_hist23:22:112
2006699873,34cyclictest30834-21taskset23:09:272
2006699873,34cyclictest30702-21fschecks_count21:07:292
2006699873,34cyclictest16387-21idleruntime21:37:332
2006699873,33cyclictest2015-21grep19:42:442
2006699873,32cyclictest7121-21cron00:32:082
2006699873,32cyclictest6173-21fschecks_count19:57:312
2006699873,32cyclictest26173-21proc_pri20:57:432
2006699873,30cyclictest4042-21/usr/sbin/munin21:17:432
2006699872,37cyclictest23970-21/usr/sbin/munin20:52:412
2006699872,36cyclictest26953-21diskmemload22:17:542
2006699872,34cyclictest26447-21unixbench_multi19:17:462
2006699872,34cyclictest26447-21unixbench_multi19:17:462
2006699872,33cyclictest29990-21latency_hist23:07:132
20061998762,9cyclictest181rcu_preempt00:01:111
20061998759,11cyclictest181rcu_preempt20:57:131
20056998753,24cyclictest7390-21latency_hist00:32:120
2006699864,32cyclictest9171-21/usr/sbin/munin20:07:242
2006699864,31cyclictest352-21runrttasks21:32:242
2006699864,29cyclictest7859-21/usr/sbin/munin22:22:472
2006699863,36cyclictest7653-21ls20:02:272
2006699863,34cyclictest2988-21tune2fs19:47:312
20066998633,44cyclictest0-21swapper/223:42:452
2006699863,32cyclictest20085-21grep20:37:462
2006699863,23cyclictest30590-21find00:12:422
2006699862,36cyclictest30100-21wc22:02:452
2006699862,33cyclictest988-21latency_hist19:42:122
2005699869,66cyclictest27108-21apt-get23:02:330
20056998659,21cyclictest26902-21apt-key23:02:130
20056998616,34cyclictest14478-21munin-run22:37:090
20056998616,34cyclictest14478-21munin-run22:37:090
20056998613,51cyclictest352-21runrttasks21:56:030
2007499856,61cyclictest0-21swapper/323:02:273
2006699855,31cyclictest2417-21/usr/sbin/munin00:22:422
2006699855,31cyclictest2417-21/usr/sbin/munin00:22:422
2006699854,34cyclictest25154-21grep21:53:272
2006699853,32cyclictest352-21runrttasks21:51:442
2006699853,32cyclictest1498-21latency_hist22:12:122
2006699853,32cyclictest11550-21latency_hist22:32:112
2006699852,38cyclictest21756-21ssh22:52:132
2006199858,67cyclictest19478-21idleruntime-cro23:52:091
20056998557,19cyclictest19676-21snmpd23:22:110
20056998523,28cyclictest1240-21/usr/sbin/munin21:12:270
20056998523,28cyclictest1240-21/usr/sbin/munin21:12:270
20056998512,36cyclictest352-21runrttasks22:04:380
2006699844,34cyclictest22567-21find22:52:422
2006699844,29cyclictest277-21dbus-daemon22:42:102
2006699844,28cyclictest24552-21mailstats19:12:412
2006699843,33cyclictest9732-21/usr/sbin/munin23:32:372
2006699843,32cyclictest1719-21/usr/sbin/munin22:12:362
2006699843,32cyclictest1664-21cat21:12:382
2006699843,32cyclictest1664-21cat21:12:382
2005699848,36cyclictest19503-21ssh21:42:360
2005699848,36cyclictest19503-21ssh21:42:360
20056998451,23cyclictest2436-21apt-get00:22:210
20056998451,23cyclictest2436-21apt-get00:22:210
20056998419,24cyclictest10561-21ssh22:29:400
20056998417,27cyclictest18410-21ssh21:41:200
20056998410,51cyclictest3668-21rm22:15:050
2006699834,28cyclictest32535-21if_eth023:12:322
2006699834,28cyclictest32535-21if_eth023:12:322
2006699833,34cyclictest352-21runrttasks20:25:122
2006699833,34cyclictest352-21runrttasks20:25:122
2006699833,30cyclictest559-21ssh00:17:492
2006699833,30cyclictest28459-21cron19:27:092
2006699833,30cyclictest28459-21cron19:27:092
20066998313,23cyclictest27182-21sh21:57:382
20066998313,23cyclictest27182-21sh21:57:382
20061998354,21cyclictest28467-21kworker/u19:2+events_unbound19:37:091
20061998328,19cyclictest0-21swapper/119:39:071
20056998352,22cyclictest22283-21apt-get19:07:280
20056998322,21cyclictest2160-21ssh23:16:350
20056998322,21cyclictest2160-21ssh23:16:350
1915628321,40sleep30-21swapper/318:57:513
2006699824,32cyclictest352-21runrttasks23:01:202
20061998255,7cyclictest181rcu_preempt22:42:231
20061998250,9cyclictest181rcu_preempt21:47:121
20061998250,9cyclictest181rcu_preempt21:47:121
20061998249,12cyclictest181rcu_preempt19:07:211
20061998238,14cyclictest181rcu_preempt21:27:371
20056998259,15cyclictest14062-21apt-key20:22:140
20056998219,22cyclictest25261-21idleruntime-cro19:17:100
20056998218,27cyclictest26953-21diskmemload21:32:470
20056998218,26cyclictest31977-21apt-get19:37:210
20056998217,24cyclictest32716-21latency_hist00:17:120
20061998152,10cyclictest181rcu_preempt23:52:481
20061998151,15cyclictest181rcu_preempt23:11:331
20061998142,13cyclictest181rcu_preempt21:07:211
20056998116,27cyclictest23768-21taskset00:00:070
2006699803,31cyclictest19564-21ssh23:52:122
2006699802,37cyclictest352-21runrttasks00:00:282
20061998062,12cyclictest277-21dbus-daemon23:47:091
20061998059,14cyclictest2374-21systemd-journal21:59:581
20061998059,14cyclictest2374-21systemd-journal21:59:581
20061998051,10cyclictest181rcu_preempt20:06:571
20061998036,10cyclictest181rcu_preempt21:52:211
20061998031,8cyclictest181rcu_preempt22:12:281
2005699809,28cyclictest0-21swapper/019:32:320
20056998035,25cyclictest26469-21sh00:04:180
20056998028,19cyclictest9171-21/usr/sbin/munin20:07:420
20056998021,27cyclictest28629-21ssh00:09:400
20056998021,27cyclictest28629-21ssh00:09:400
20056998021,19cyclictest5155-21/usr/sbin/munin21:17:480
20056998018,29cyclictest6774-21rm22:22:220
20056998018,26cyclictest0-21swapper/023:41:090
20056998017,26cyclictest11764-21ssh21:29:340
20056998015,28cyclictest22972-21/usr/sbin/munin20:47:420
20056998012,25cyclictest0-21swapper/019:47:330
20061997948,9cyclictest181rcu_preempt20:57:221
20061997948,12cyclictest181rcu_preempt21:35:001
20061997934,8cyclictest181rcu_preempt22:02:261
20061997930,15cyclictest181rcu_preempt20:32:111
2005699796,42cyclictest0-21swapper/022:02:120
2005699796,42cyclictest0-21swapper/022:02:120
20056997952,19cyclictest25536-21latency_hist20:57:130
20056997951,19cyclictest6593-21ssh22:22:130
20056997921,25cyclictest31487-21sed21:07:450
20056997918,25cyclictest27027-21latency_hist19:22:120
20056997918,25cyclictest27027-21latency_hist19:22:120
20056997917,23cyclictest30649-21ssh23:08:570
20056997911,27cyclictest21863-21taskset22:52:210
20061997862,11cyclictest20047-21cyclictest21:13:171
20061997862,11cyclictest20047-21cyclictest21:13:171
20061997850,18cyclictest26953-21diskmemload23:12:311
20061997850,18cyclictest26953-21diskmemload23:12:311
20061997849,8cyclictest181rcu_preempt23:00:211
20061997829,20cyclictest14125-21apt-config20:22:131
20056997818,27cyclictest21463-21sh22:52:030
20056997816,25cyclictest9118-21ssh23:31:220
20056997816,25cyclictest9118-21ssh23:31:220
20056997813,26cyclictest812-21ssh00:18:310
20056997811,28cyclictest19085-21cut20:37:130
20061997734,12cyclictest0-21swapper/120:17:121
2005699779,27cyclictest19238-21cstates20:37:270
20056997758,14cyclictest20047-21cyclictest23:23:140
20056997716,25cyclictest21318-21/usr/sbin/munin20:42:410
20056997716,24cyclictest25922-21cat20:57:340
20056997715,28cyclictest27685-21/usr/sbin/munin19:22:410
20056997715,28cyclictest27685-21/usr/sbin/munin19:22:410
20056997713,25cyclictest0-21swapper/021:03:170
20056997713,25cyclictest0-21swapper/021:03:170
20061997660,12cyclictest20047-21cyclictest22:22:271
20061997658,10cyclictest0-21swapper/119:27:091
20061997658,10cyclictest0-21swapper/119:27:091
20061997648,12cyclictest181rcu_preempt00:28:311
20061997648,11cyclictest181rcu_preempt19:52:121
20061997645,11cyclictest181rcu_preempt22:42:121
20061997640,11cyclictest0-21swapper/119:12:331
20056997623,22cyclictest0-21swapper/020:02:470
20061997552,9cyclictest181rcu_preempt19:07:131
20061997551,16cyclictest23890-21latency_hist20:52:121
20061997543,12cyclictest181rcu_preempt21:03:311
20061997543,12cyclictest181rcu_preempt21:03:311
20061997538,9cyclictest181rcu_preempt23:22:351
20061997529,9cyclictest181rcu_preempt23:32:381
20056997516,20cyclictest16779-21ssh22:42:170
20056997514,21cyclictest2399-21seq19:46:260
20061997440,11cyclictest181rcu_preempt22:29:181
20061997435,13cyclictest0-21swapper/120:42:271
20061997417,11cyclictest0-21swapper/100:24:451
20061997417,11cyclictest0-21swapper/100:24:451
20056997419,22cyclictest20716-21netstat19:02:360
20056997410,29cyclictest6693-21sed19:57:440
20056997410,28cyclictest12247-21latency_hist20:17:120
20061997339,11cyclictest0-21swapper/122:51:541
20061997335,12cyclictest181rcu_preempt21:21:451
20061997331,12cyclictest181rcu_preempt19:57:351
20056997314,24cyclictest5942-21cut19:57:130
20056997313,26cyclictest352-21runrttasks20:28:560
55202720,8sleep10-21swapper/122:19:361
20061997252,14cyclictest300-21rs:main1
20061997249,16cyclictest297-21systemd-logind00:22:101
20061997245,17cyclictest27050-21latency_hist19:22:121
20061997245,17cyclictest27050-21latency_hist19:22:121
20061997237,11cyclictest181rcu_preempt21:37:351
20056997212,24cyclictest4873-21tr19:52:370
199642728,36sleep019979-21munin-run19:02:120
20061997141,11cyclictest181rcu_preempt20:23:291
20061997141,11cyclictest181rcu_preempt20:23:291
20061997130,10cyclictest181rcu_preempt22:52:541
20061997054,10cyclictest19304-21/usr/sbin/munin20:37:311
2006199704,39cyclictest1128-21apt-get19:42:191
20061997043,11cyclictest181rcu_preempt22:32:231
20061997043,11cyclictest181rcu_preempt22:32:231
20061997027,8cyclictest181rcu_preempt19:52:321
20061997027,8cyclictest181rcu_preempt19:52:321
2006199702,47cyclictest30329-21latency_hist19:32:131
20061996938,9cyclictest181rcu_preempt20:07:331
20061996925,13cyclictest0-21swapper/100:06:541
20061996921,27cyclictest0-21swapper/121:22:441
20061996817,12cyclictest181rcu_preempt23:37:481
2006199673,43cyclictest29541-21idleruntime-cro00:12:091
2006199673,43cyclictest29541-21idleruntime-cro00:12:091
20061996716,8cyclictest18256-21switchtime20:32:451
2006199662,43cyclictest31-21ksoftirqd/121:51:291
2007499623,35cyclictest0-21swapper/320:22:373
2007499623,35cyclictest0-21swapper/320:22:373
2007499584,41cyclictest22582-21taskset23:57:113
2007499584,40cyclictest32001-21chrt19:37:253
2007499573,40cyclictest27553-21chrt21:58:393
2007499573,40cyclictest27553-21chrt21:58:393
2007499563,38cyclictest27889-21chrt00:07:363
2007499563,38cyclictest27889-21chrt00:07:363
2007499563,35cyclictest5402-21taskset00:27:173
2007499563,35cyclictest30047-21chrt23:07:153
20074995620,31cyclictest0-21swapper/323:52:093
2007499554,37cyclictest7101-21taskset23:26:243
2007499543,40cyclictest12599-21taskset20:17:313
2007499543,35cyclictest19047-21taskset21:42:233
2007499543,35cyclictest19047-21taskset21:42:233
2007499543,34cyclictest19795-21chrt22:47:213
2007499534,34cyclictest9095-21chrt23:31:183
2007499534,34cyclictest9095-21chrt23:31:183
2007499533,36cyclictest8043-21chrt21:22:453
2007499533,36cyclictest12574-21chrt22:32:433
2007499533,36cyclictest12574-21chrt22:32:433
2007499523,40cyclictest23359-21chrt23:58:573
2007499523,39cyclictest32556-21chrt22:08:363
2007499523,39cyclictest25762-21taskset21:54:183
2007499523,38cyclictest28750-21chrt19:27:253
2007499523,38cyclictest17270-21chrt20:32:093
2007499523,37cyclictest28458-21chrt19:27:093
2007499523,37cyclictest28458-21chrt19:27:093
2007499523,36cyclictest2040-21chrt19:42:443
20074995225,8cyclictest0-21swapper/321:02:563
20074995225,8cyclictest0-21swapper/321:02:563
2007499522,38cyclictest25731-21taskset20:57:283
2007499522,37cyclictest5717-21chrt21:19:213
2007499513,38cyclictest5176-21taskset19:52:443
2007499513,38cyclictest5176-21taskset19:52:443
2007499513,36cyclictest2587-21chrt23:17:123
2007499513,36cyclictest2587-21chrt23:17:123
2007499513,34cyclictest5872-21taskset22:20:353
2007499512,37cyclictest32554-21taskset00:17:083
2007499512,37cyclictest1729-21chrt22:12:213
2007499512,37cyclictest17287-21taskset23:46:503
2007499512,36cyclictest3701-21chrt00:23:123
2007499512,36cyclictest3701-21chrt00:23:123
2007499512,36cyclictest3111-21chrt19:47:343
2007499504,33cyclictest22363-21chrt19:07:293
2007499503,36cyclictest23126-21chrt20:47:443
2007499503,35cyclictest30623-21taskset21:07:253
2007499503,32cyclictest26818-21chrt23:02:123
2007499503,30cyclictest15875-21chrt21:37:123
2007499502,36cyclictest20457-21taskset19:02:313
2007499502,34cyclictest18890-21taskset22:45:313
2007499494,31cyclictest26380-21taskset19:17:453
2007499494,31cyclictest26380-21taskset19:17:453
2007499493,36cyclictest10044-21chrt23:32:313
2007499493,35cyclictest7719-21chrt20:02:283
2007499493,35cyclictest16882-21chrt21:37:453
2007499493,33cyclictest8156-21chrt22:23:353
2007499493,32cyclictest22830-21chrt21:49:363
2007499492,33cyclictest31750-21chrt22:07:023
2007499492,31cyclictest12977-21chrt21:32:133
2007499483,33cyclictest30492-21taskset19:32:273
2007499482,37cyclictest14555-21chrt23:42:123
2007499482,35cyclictest25386-21taskset19:17:133
2007499482,33cyclictest10988-21taskset20:12:293
2007499482,32cyclictest17530-21taskset20:32:213
2007499474,34cyclictest26790-21chrt00:05:133
2007499473,35cyclictest3060-21chrt23:18:253
2007499473,34cyclictest24241-21chrt22:56:213
2007499464,31cyclictest9730-21chrt20:07:393
2007499463,31cyclictest10899-21chrt22:30:363
2007499463,28cyclictest7476-21chrt20:02:123
2007499452,33cyclictest20906-21chrt20:42:303
2007499452,30cyclictest3241-21taskset21:15:483
2007499452,30cyclictest3241-21taskset21:15:483
2007499443,31cyclictest25423-21chrt20:57:123
2007499424,29cyclictest557-21taskset00:17:493
2007499422,31cyclictest15183-21chrt22:38:303
2007499412,27cyclictest19139-21taskset20:37:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional