You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 04:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Wed Apr 29, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25928211613,67sleep125930-21/usr/sbin/munin18:57:321
274009911336,12cyclictest0-21swapper/223:12:372
27400991064,41cyclictest11384-21fschecks_time23:57:362
27400999913,65cyclictest10504-21apt-get23:02:292
2739399993,84cyclictest352-21runrttasks22:37:400
2739399993,84cyclictest352-21runrttasks22:37:400
2740099954,71cyclictest27666-21apt-get22:37:292
2740099954,71cyclictest27666-21apt-get22:37:292
27393999575,12cyclictest12095-21grep23:57:500
27400999367,18cyclictest8078-21/usr/sbin/munin23:52:312
27397999372,8cyclictest0-21swapper/123:02:331
2739799933,67cyclictest24441-21apt-get22:32:331
27400999058,23cyclictest18037-21ssh00:07:502
2739799904,75cyclictest23341-21apt-get00:17:341
271372906,66sleep00-21swapper/019:00:530
27400998962,18cyclictest29700-21ssh21:47:372
2723628914,50sleep227259-21munin-run19:02:142
27400998872,9cyclictest0-21swapper/223:47:142
27400998872,9cyclictest0-21swapper/223:47:142
27400998745,10cyclictest181rcu_preempt22:52:262
2740099874,29cyclictest24915-21latency_hist20:32:172
2740099874,29cyclictest24915-21latency_hist20:32:172
27393998737,28cyclictest15-21ksoftirqd/019:37:490
27393998732,32cyclictest15-21ksoftirqd/020:47:510
2740099865,68cyclictest17705-21kworker/u18:1+ext4-rsv-conversion00:17:372
2740099863,73cyclictest12184-21apt-get22:12:292
2740099863,73cyclictest12184-21apt-get22:12:292
2739399863,59cyclictest23413-21modprobe00:17:350
27393998565,14cyclictest1970-21latency_hist19:22:170
27400998464,14cyclictest28186-21latency_hist23:32:172
27400998460,15cyclictest15049-21mailstats21:22:492
27400998450,13cyclictest181rcu_preempt19:37:492
27397998437,17cyclictest1945-21diskmemload23:17:391
27397998437,17cyclictest1945-21diskmemload23:17:391
2739399846,68cyclictest12217-21ssh22:12:290
2739399846,68cyclictest12217-21ssh22:12:290
27393998429,31cyclictest15-21ksoftirqd/020:57:360
27393998429,31cyclictest15-21ksoftirqd/020:57:360
27400998360,15cyclictest352-21runrttasks00:12:312
27393998363,14cyclictest24132-21latency_hist22:32:160
27400998261,16cyclictest909-21latency_hist22:47:162
27400998261,13cyclictest16904-21latency_hist00:07:162
27397998261,14cyclictest17396-21fschecks_time00:07:361
27397998260,12cyclictest0-21swapper/119:12:131
2739799821,38cyclictest0-21swapper/123:07:421
27400998142,17cyclictest181rcu_preempt20:37:302
27400998142,17cyclictest181rcu_preempt20:37:302
27397998160,14cyclictest614-21fschecks_count19:17:371
27397998159,14cyclictest9949-21latency_hist23:02:161
27397998159,14cyclictest27382-21cyclictest19:26:081
27397998157,17cyclictest13629-21fschecks_count19:57:351
27393998164,11cyclictest2374-21systemd-journal23:04:320
27400998052,10cyclictest181rcu_preempt21:32:272
27400998052,10cyclictest181rcu_preempt21:32:272
27397998056,16cyclictest2374-21systemd-journal21:29:321
2739399803,51cyclictest15-21ksoftirqd/000:26:320
27400997958,14cyclictest1945-21diskmemload22:24:322
27400997958,14cyclictest1945-21diskmemload22:24:322
27400997936,11cyclictest181rcu_preempt20:12:502
27400997931,12cyclictest181rcu_preempt23:49:572
27400997912,30cyclictest2245-21grep22:47:502
27393997938,34cyclictest0-21swapper/023:37:420
27393997936,14cyclictest181rcu_preempt23:29:330
27400997836,14cyclictest0-21swapper/219:17:492
27397997861,12cyclictest2374-21systemd-journal22:39:321
27397997861,12cyclictest2374-21systemd-journal22:39:321
27393997860,12cyclictest352-21runrttasks19:47:400
27393997859,13cyclictest19357-21latency_hist23:17:160
27393997844,28cyclictest544-21cron19:07:140
27393997844,28cyclictest544-21cron19:07:140
2739399782,34cyclictest10258-21cat23:02:170
2739399781,46cyclictest15-21ksoftirqd/022:02:490
27397997757,12cyclictest27382-21cyclictest19:40:041
2739799772,61cyclictest7448-21kworker/u19:1+ext4-rsv-conversion22:13:271
2739799772,61cyclictest7448-21kworker/u19:1+ext4-rsv-conversion22:13:271
27397997721,17cyclictest0-21swapper/121:32:401
27397997721,17cyclictest0-21swapper/121:32:401
27393997761,10cyclictest1945-21diskmemload23:21:250
27393997761,10cyclictest1945-21diskmemload23:21:250
27393997757,14cyclictest32258-21latency_hist21:52:170
27393997753,8cyclictest181rcu_preempt21:43:240
2739399773,44cyclictest15-21ksoftirqd/022:12:180
2739399773,44cyclictest15-21ksoftirqd/022:12:180
27400997656,13cyclictest27382-21cyclictest00:28:062
27400997637,10cyclictest0-21swapper/222:36:142
27397997657,12cyclictest31241-21kworker/u17:3+rpciod23:57:151
27397997655,15cyclictest17071-21latency_hist21:27:151
27397997654,12cyclictest12425-21kworker/u17:2-rpciod22:45:261
27397997652,14cyclictest0-21swapper/123:31:451
27397997619,24cyclictest1945-21diskmemload22:24:011
27397997619,24cyclictest1945-21diskmemload22:24:011
27393997659,12cyclictest20653-21fschecks_time21:32:370
27393997659,12cyclictest20653-21fschecks_time21:32:370
27393997656,14cyclictest1945-21diskmemload23:07:440
27393997654,8cyclictest181rcu_preempt00:03:340
27393997648,20cyclictest4943-21cron19:32:140
27393997633,22cyclictest181rcu_preempt00:07:400
27393997611,36cyclictest15-21ksoftirqd/020:55:590
27400997548,19cyclictest32358-21switchtime23:37:482
2740099753,45cyclictest13578-21sh23:07:302
27400997528,17cyclictest26316-21munin-plugin-st20:37:132
27400997528,17cyclictest26316-21munin-plugin-st20:37:132
2739799757,23cyclictest0-21swapper/119:51:561
27397997549,18cyclictest27382-21cyclictest19:52:421
27397997523,13cyclictest0-21swapper/121:40:191
27397997517,24cyclictest0-21swapper/122:52:471
27397997516,22cyclictest18045-21sh22:22:161
27397997511,29cyclictest26345-21unin-run20:37:141
27397997511,29cyclictest26345-21unin-run20:37:141
27397997510,26cyclictest12664-21seq23:59:101
27393997559,11cyclictest58-21kcompactd020:30:290
27393997559,11cyclictest58-21kcompactd020:30:290
27393997557,13cyclictest8646-21fschecks_count19:42:360
27393997557,11cyclictest297-21systemd-logind20:27:140
27393997530,22cyclictest0-21swapper/021:02:530
27400997436,10cyclictest0-21swapper/221:52:302
27397997456,12cyclictest11679-21systemd-run22:12:151
27397997456,12cyclictest11679-21systemd-run22:12:151
27397997413,24cyclictest4516-21runrttasks21:59:141
27397997413,24cyclictest4516-21runrttasks21:59:141
27393997458,11cyclictest27382-21cyclictest21:12:320
27393997452,8cyclictest181rcu_preempt23:43:000
27393997452,8cyclictest181rcu_preempt23:43:000
27393997451,15cyclictest2374-21systemd-journal20:07:330
27393997449,20cyclictest7448-21kworker/u19:1-rpciod21:57:200
27393997449,20cyclictest7448-21kworker/u19:1-rpciod21:57:200
27393997444,24cyclictest0-21swapper/021:52:420
27393997434,20cyclictest181rcu_preempt22:26:020
27393997434,20cyclictest181rcu_preempt22:26:020
27393997414,30cyclictest15-21ksoftirqd/020:38:100
27393997414,30cyclictest15-21ksoftirqd/020:38:100
27393997412,32cyclictest15-21ksoftirqd/021:09:170
27393997412,32cyclictest15-21ksoftirqd/021:09:170
2739799738,28cyclictest27739-21seq00:23:381
27397997351,14cyclictest1945-21diskmemload21:02:181
27397997347,16cyclictest2816-21rm21:57:171
27397997310,24cyclictest9785-21seq19:46:301
27393997347,8cyclictest181rcu_preempt21:17:210
2739399733,29cyclictest15-21ksoftirqd/021:38:520
27393997325,33cyclictest0-21swapper/000:27:460
27393997319,17cyclictest0-21swapper/022:34:160
2740699726,47cyclictest0-21swapper/322:12:293
2740699726,47cyclictest0-21swapper/322:12:293
27400997254,11cyclictest27170-21ssh21:42:502
27400997242,8cyclictest0-21swapper/220:42:362
2739799728,46cyclictest7843-21latency_hist23:52:151
2739799725,36cyclictest3921-21latency_hist22:52:151
27397997251,13cyclictest0-21swapper/120:27:351
27397997251,13cyclictest0-21swapper/120:27:351
27397997231,27cyclictest16775-21awk00:07:131
27397997211,23cyclictest28669-21seq19:04:171
27397997211,23cyclictest28669-21seq19:04:171
27393997246,11cyclictest181rcu_preempt23:53:060
2739399722,40cyclictest15-21ksoftirqd/020:00:390
2739399722,32cyclictest15-21ksoftirqd/020:32:400
2739399722,32cyclictest15-21ksoftirqd/020:32:400
2739399722,28cyclictest15886-21processes22:17:470
2740099718,45cyclictest5606-21latency_hist22:02:162
2740099718,45cyclictest5606-21latency_hist22:02:162
27400997154,12cyclictest27382-21cyclictest21:39:432
2739799717,23cyclictest29252-21seq20:43:441
27397997153,12cyclictest277-21dbus-daemon23:27:141
2739799714,45cyclictest20092-21/usr/sbin/munin00:12:341
27397997123,12cyclictest0-21swapper/121:19:341
27397997110,23cyclictest16268-21seq20:06:021
27393997153,11cyclictest21118-21ssh00:12:490
27393997152,14cyclictest58-21kcompactd020:03:320
27393997130,17cyclictest181rcu_preempt22:49:480
2740699708,43cyclictest0-21swapper/300:17:353
2740099705,22cyclictest15180-21sshd19:42:462
27400997049,12cyclictest58-21kcompactd021:07:162
27400997043,15cyclictest0-21swapper/223:32:512
27400997021,15cyclictest181rcu_preempt22:27:282
27397997045,19cyclictest9267-21timerwakeupswit21:12:511
27397997042,19cyclictest1922-21modprobe21:02:131
27397997042,19cyclictest1922-21modprobe21:02:131
2739799703,50cyclictest16875-21apt-get20:07:471
27397997012,25cyclictest22935-21seq20:25:341
27393997051,14cyclictest5260-21cpu19:32:300
27393997051,14cyclictest5260-21cpu19:32:300
27393997047,8cyclictest181rcu_preempt21:23:450
2739399703,45cyclictest15-21ksoftirqd/022:52:240
27393997034,12cyclictest181rcu_preempt19:07:250
27393997034,10cyclictest181rcu_preempt20:18:130
27400996946,14cyclictest0-21swapper/223:00:392
27400996945,8cyclictest0-21swapper/219:52:402
2740099693,35cyclictest23465-21cat23:22:502
27400996924,14cyclictest16234-21grep22:17:512
2740099691,27cyclictest6166-21fschecks_count22:02:352
2739799696,24cyclictest0-21swapper/120:17:421
27397996953,10cyclictest26657-21fschecks_time21:42:371
27397996947,15cyclictest300-21rs:main1
2739799693,45cyclictest27553-21unixbench_singl20:37:511
2739799693,45cyclictest27553-21unixbench_singl20:37:511
27397996914,23cyclictest30048-21seq23:34:021
27393996954,11cyclictest27382-21cyclictest20:47:150
27393996934,8cyclictest181rcu_preempt23:22:330
27393996919,11cyclictest58-21kcompactd023:48:390
27406996821,29cyclictest0-21swapper/322:37:413
27406996821,29cyclictest0-21swapper/322:37:413
27400996852,9cyclictest0-21swapper/221:17:522
2740099684,43cyclictest29326-21latency19:07:222
2739799688,26cyclictest32519-21seq20:54:011
27397996849,12cyclictest17529-21unixbench_multi23:12:481
27397996846,13cyclictest0-21swapper/100:27:501
27393996848,11cyclictest0-21swapper/022:42:370
27393996832,18cyclictest29320-21ssh23:32:500
27393996829,16cyclictest181rcu_preempt21:32:020
27400996751,10cyclictest1601-21kworker/u17:1-rpciod21:27:412
27400996748,12cyclictest0-21swapper/221:07:522
27400996748,12cyclictest0-21swapper/221:07:522
27400996747,15cyclictest22098-21kernelversion20:22:382
27400996724,13cyclictest181rcu_preempt21:12:332
27397996734,23cyclictest6783-21timerandwakeup22:02:491
27400996639,19cyclictest21614-21latency_hist20:22:162
27400996636,12cyclictest181rcu_preempt00:22:512
27400996620,8cyclictest0-21swapper/219:22:452
27397996646,14cyclictest30385-21unixbench_singl21:47:511
27397996637,23cyclictest21514-21fschecks_time22:27:361
2739799663,45cyclictest4573-21grep19:27:521
27397996621,12cyclictest6379-21users21:07:531
27397996621,12cyclictest6379-21users21:07:531
27397996611,23cyclictest509-21seq23:38:481
27393996644,8cyclictest181rcu_preempt19:22:250
2739399663,47cyclictest18761-21fschecks_time20:12:360
27393996617,13cyclictest15-21ksoftirqd/019:13:020
27400996542,13cyclictest0-21swapper/219:12:352
27400996537,15cyclictest0-21swapper/220:57:172
2739799656,36cyclictest352-21runrttasks20:49:191
27397996542,12cyclictest0-21swapper/123:42:231
27397996542,12cyclictest0-21swapper/123:42:231
27397996510,24cyclictest19666-21seq20:14:491
2740099649,28cyclictest0-21swapper/219:02:302
2740099649,28cyclictest0-21swapper/219:02:302
27400996444,13cyclictest0-21swapper/222:07:362
27400996444,13cyclictest0-21swapper/222:07:362
2740099644,33cyclictest3598-21/usr/sbin/munin19:27:352
27400996437,10cyclictest181rcu_preempt19:57:322
27400996434,9cyclictest181rcu_preempt20:08:372
27397996418,12cyclictest5787-21ntp_states19:32:451
27397996418,12cyclictest5787-21ntp_states19:32:451
2739399643,43cyclictest352-21runrttasks19:55:040
2620526419,24sleep30-21swapper/318:57:423
2740099634,39cyclictest16705-21latency_hist20:07:172
27400996340,16cyclictest10481-21kernelversion19:47:392
27400996243,13cyclictest0-21swapper/223:22:142
27400996243,13cyclictest0-21swapper/223:22:142
2740099623,38cyclictest839-21fschecks_time20:57:362
2740099623,38cyclictest839-21fschecks_time20:57:362
27400996118,9cyclictest181rcu_preempt20:47:322
2740099603,40cyclictest6628-21latency_hist19:37:152
2740099603,40cyclictest6628-21latency_hist19:37:152
182842560,7sleep30-21swapper/322:22:313
182842560,7sleep30-21swapper/322:22:313
53062550,9sleep30-21swapper/322:52:513
2740699543,42cyclictest14488-21taskset23:07:503
2740699543,39cyclictest29012-21chrt21:47:133
2740699533,35cyclictest8574-21chrt19:42:323
2740699533,31cyclictest21668-21chrt20:22:173
2740699524,39cyclictest1863-21taskset23:42:153
2740699523,39cyclictest17908-21chrt23:13:383
2740699523,38cyclictest29264-21taskset21:47:173
2740699522,37cyclictest26362-21chrt23:27:493
2740699514,36cyclictest17210-21chrt20:08:323
2740699513,35cyclictest13586-21chrt00:01:453
2740699513,34cyclictest31348-21chrt19:12:313
2740699513,34cyclictest24054-21chrt23:24:083
2740699513,33cyclictest20365-21chrt00:12:353
2740699512,36cyclictest31989-21chrt00:32:123
2740699512,35cyclictest30912-21chrt22:42:373
2740699512,35cyclictest28401-21chrt19:02:513
2740699512,35cyclictest28401-21chrt19:02:513
2740699504,32cyclictest18820-21chrt20:12:373
2740699503,37cyclictest7507-21taskset19:37:463
2740699503,35cyclictest8023-21taskset22:05:373
2740699503,35cyclictest15091-21taskset00:02:503
2740699503,34cyclictest10733-21chrt19:47:463
2740699503,33cyclictest14171-21chrt21:22:173
2740699503,31cyclictest2025-21taskset22:47:483
2740699502,36cyclictest778-21chrt21:52:483
2740699502,36cyclictest649-21taskset19:17:373
2740699493,35cyclictest5763-21chrt22:02:163
2740699493,35cyclictest5763-21chrt22:02:163
2740699493,35cyclictest19279-21chrt00:10:523
2740699493,33cyclictest21269-21chrt23:19:423
2740699493,33cyclictest21269-21chrt23:19:423
2740699493,32cyclictest17331-21chrt22:20:453
2740699493,31cyclictest9072-21chrt21:12:483
2740699493,31cyclictest11206-21chrt22:11:083
2740699493,31cyclictest11206-21chrt22:11:083
2740699492,38cyclictest24065-21taskset22:32:163
2740699492,37cyclictest7134-21chrt21:09:563
2740699492,37cyclictest7134-21chrt21:09:563
2740699492,36cyclictest8544-21chrt23:53:103
2740699492,36cyclictest3353-21taskset23:43:213
2740699492,36cyclictest3353-21taskset23:43:213
2740699492,35cyclictest22771-21chrt21:36:353
2740699492,35cyclictest22771-21chrt21:36:353
2740699492,35cyclictest14098-21chrt19:57:473
2740699492,32cyclictest2470-21chrt21:02:353
2740699483,35cyclictest6606-21chrt19:37:133
2740699483,35cyclictest6606-21chrt19:37:133
2740699483,35cyclictest2145-21chrt19:22:313
2740699483,32cyclictest5481-21chrt23:47:383
2740699483,32cyclictest31190-21chrt20:52:143
2740699483,32cyclictest15123-21chrt21:22:503
2740699483,32cyclictest12325-21chrt19:52:443
2740699482,32cyclictest28274-21chrt20:42:303
2740699482,32cyclictest11467-21chrt23:03:123
2740699473,34cyclictest27451-21chrt00:22:523
2740699473,33cyclictest29554-21chrt23:32:513
2740699473,31cyclictest26416-21chrt20:37:153
2740699473,31cyclictest26416-21chrt20:37:153
2740699463,32cyclictest24094-21chrt20:27:483
2740699463,32cyclictest24094-21chrt20:27:483
2740699463,31cyclictest22319-21chrt20:22:463
2740699453,33cyclictest24311-21chrt21:37:493
2740699452,30cyclictest24458-21chrt22:32:283
2740699442,33cyclictest8847-21chrt22:59:113
2740699433,27cyclictest871-21chrt20:57:373
2740699433,27cyclictest871-21chrt20:57:373
2740699423,29cyclictest30814-21chrt19:10:323
2740699412,29cyclictest31600-21taskset20:52:343
2740699394,26cyclictest19525-21chrt21:31:003
2740699383,25cyclictest15579-21chrt20:02:453
2740699382,27cyclictest4903-21taskset19:31:423
2740699372,25cyclictest27568-21chrt20:37:523
2740699372,25cyclictest27568-21chrt20:37:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional