You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 06:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Mon May 04, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31692212513,77sleep031730-21grep18:57:190
561991192,81cyclictest30100-21apt-get22:32:081
31518210919,42sleep20-21swapper/218:57:052
3173421089,73sleep131789-21/usr/sbin/munin18:57:201
561991023,69cyclictest19645-21apt-get19:57:081
5679910132,26cyclictest14946-21runrttasks22:59:132
5619910070,9cyclictest181rcu_preempt23:27:171
5619910070,9cyclictest181rcu_preempt23:27:171
561999547,9cyclictest181rcu_preempt20:52:141
567999444,39cyclictest31206-21apt-get20:32:042
561999452,14cyclictest181rcu_preempt00:27:271
561999452,14cyclictest181rcu_preempt00:27:271
561999447,10cyclictest181rcu_preempt00:06:561
54999943,67cyclictest30663-21apt-get22:32:090
561999348,12cyclictest181rcu_preempt00:26:541
56199933,63cyclictest4119-21apt-get19:12:101
56199933,63cyclictest4119-21apt-get19:12:101
561999331,31cyclictest0-21swapper/121:07:281
567999235,19cyclictest8184-21diskmemload23:26:202
567999228,25cyclictest352-21runrttasks23:27:052
567999228,25cyclictest352-21runrttasks23:27:052
561999263,19cyclictest3679-21sh21:47:171
561999252,25cyclictest181rcu_preempt19:22:181
567999129,21cyclictest12083-21seq23:48:012
549999138,15cyclictest0-21swapper/021:47:280
567999031,22cyclictest27409-21rm00:13:502
567999030,24cyclictest24398-21/usr/sbin/munin22:22:172
567999029,26cyclictest2806-21ssh21:46:192
567999026,32cyclictest8184-21diskmemload22:42:192
567999026,32cyclictest8184-21diskmemload22:42:192
567999026,25cyclictest16002-21ssh22:07:222
561999065,17cyclictest18704-21fschecks_time22:12:141
561999063,8cyclictest181rcu_preempt21:32:311
57499893,49cyclictest0-21swapper/321:12:213
567998931,46cyclictest0-21swapper/222:17:062
567998928,28cyclictest22756-21sh00:06:572
567998928,21cyclictest8020-21seq21:54:012
567998926,24cyclictest20618-21/usr/sbin/munin00:02:292
567998925,25cyclictest7841-21/usr/sbin/munin19:22:262
561998952,14cyclictest181rcu_preempt23:32:281
561998950,13cyclictest181rcu_preempt20:51:551
561998950,13cyclictest181rcu_preempt20:51:551
561998941,12cyclictest181rcu_preempt20:42:211
567998872,9cyclictest0-21swapper/221:12:312
567998830,23cyclictest30884-21seq20:31:422
567998830,20cyclictest32375-21ssh22:34:132
567998826,23cyclictest18504-21ssh23:59:072
567998821,28cyclictest10757-21runrttasks19:28:162
561998846,13cyclictest181rcu_preempt19:41:541
561998845,10cyclictest181rcu_preempt22:50:551
567998730,25cyclictest18666-21seq23:06:142
567998729,24cyclictest19676-21snmpd19:36:462
567998728,24cyclictest29269-21seq20:26:362
567998725,25cyclictest29383-21rm21:36:392
561998752,13cyclictest181rcu_preempt19:52:211
561998746,16cyclictest181rcu_preempt23:21:561
57499865,65cyclictest30715-21chrt22:32:133
567998628,27cyclictest14155-21seq19:40:142
567998628,23cyclictest8079-21seq21:00:542
567998622,28cyclictest0-21swapper/221:27:192
567998622,28cyclictest0-21swapper/221:27:192
561998646,9cyclictest181rcu_preempt20:07:111
561998640,8cyclictest181rcu_preempt22:40:281
561998634,14cyclictest21535-21ssh22:17:091
561998618,23cyclictest0-21swapper/119:27:491
549998663,16cyclictest300-21rs:main0
567998568,12cyclictest536-21cyclictest21:47:302
567998529,20cyclictest13906-21ssh21:11:082
567998528,25cyclictest9925-21ssh23:45:212
567998528,23cyclictest29495-21grep00:17:242
567998528,23cyclictest29495-21grep00:17:242
567998528,20cyclictest3809-21seq00:28:252
567998528,20cyclictest3809-21seq00:28:252
567998526,25cyclictest17343-21taskset19:49:562
567998526,25cyclictest17343-21taskset19:49:562
567998525,23cyclictest10632-21ssh22:52:182
567998525,23cyclictest10632-21ssh22:52:182
567998525,22cyclictest6288-21seq20:53:292
567998524,30cyclictest18351-21ntp_kernel_pll_21:17:212
567998524,30cyclictest18351-21ntp_kernel_pll_21:17:212
567998522,31cyclictest352-21runrttasks22:03:312
567998520,28cyclictest6550-21/usr/sbin/munin19:17:372
567998520,28cyclictest6550-21/usr/sbin/munin19:17:372
561998554,9cyclictest181rcu_preempt22:57:031
561998532,15cyclictest0-21swapper/121:26:551
567998427,22cyclictest3161-21seq20:45:032
567998427,19cyclictest1446-21seq20:38:432
567998426,22cyclictest4056-21seq19:11:182
567998426,22cyclictest19092-21seq19:56:222
567998426,18cyclictest24135-21seq20:11:012
567998423,38cyclictest352-21runrttasks21:41:082
567998423,38cyclictest352-21runrttasks21:41:082
567998423,27cyclictest0-21swapper/223:21:332
567998423,26cyclictest0-21swapper/221:22:232
561998446,10cyclictest181rcu_preempt23:37:261
561998445,10cyclictest181rcu_preempt21:57:281
561998444,16cyclictest181rcu_preempt23:47:211
567998329,23cyclictest15861-21seq19:46:342
567998327,22cyclictest8745-21ssh22:48:442
567998322,29cyclictest27383-21/usr/sbin/munin22:27:182
567998322,28cyclictest0-21swapper/220:16:312
567998321,27cyclictest0-21swapper/220:05:342
567998321,27cyclictest0-21swapper/220:05:342
561998364,14cyclictest8709-21timerandwakeup23:42:291
561998360,7cyclictest181rcu_preempt21:42:091
561998354,10cyclictest181rcu_preempt19:07:321
561998350,13cyclictest181rcu_preempt23:12:281
561998350,12cyclictest181rcu_preempt23:22:091
561998341,9cyclictest0-21swapper/120:32:031
567998265,10cyclictest31868-21kworker/u17:2-rpciod23:37:302
567998224,22cyclictest352-21runrttasks22:37:412
567998224,22cyclictest0-21swapper/222:01:062
567998219,33cyclictest26098-21/usr/sbin/munin20:17:122
561998259,8cyclictest181rcu_preempt20:16:561
561998255,11cyclictest181rcu_preempt21:31:281
561998255,11cyclictest181rcu_preempt21:31:281
561998254,9cyclictest181rcu_preempt22:07:191
561998253,8cyclictest181rcu_preempt00:12:161
561998247,7cyclictest181rcu_preempt23:58:351
54999825,40cyclictest15-21ksoftirqd/019:30:260
567998125,19cyclictest2032-21seq19:06:362
567998124,22cyclictest10784-21rm21:05:482
567998122,24cyclictest0-21swapper/223:36:082
561998159,8cyclictest181rcu_preempt22:44:491
561998159,8cyclictest181rcu_preempt22:44:491
561998157,9cyclictest181rcu_preempt20:26:571
561998148,9cyclictest181rcu_preempt22:27:271
561998148,12cyclictest181rcu_preempt19:17:491
561998145,10cyclictest181rcu_preempt22:52:291
561998145,10cyclictest181rcu_preempt22:52:291
561998141,10cyclictest181rcu_preempt22:06:551
561998133,8cyclictest181rcu_preempt00:08:411
561998132,8cyclictest181rcu_preempt23:07:191
54999813,44cyclictest15-21ksoftirqd/019:26:500
567998060,14cyclictest20203-21ntp_kernel_pll_19:58:062
567998024,23cyclictest22376-21cstates23:12:102
567998020,23cyclictest15930-21runrttasks23:55:182
561998054,9cyclictest181rcu_preempt19:47:151
561998054,9cyclictest181rcu_preempt19:47:151
561998045,13cyclictest181rcu_preempt21:12:291
561998043,11cyclictest181rcu_preempt23:06:541
561998042,12cyclictest181rcu_preempt20:26:561
567997926,21cyclictest7392-21seq19:19:242
567997913,28cyclictest0-21swapper/200:22:412
561997959,7cyclictest181rcu_preempt21:40:481
561997959,7cyclictest181rcu_preempt21:40:481
561997949,8cyclictest181rcu_preempt19:46:541
561997947,9cyclictest181rcu_preempt23:56:551
561997929,9cyclictest0-21swapper/121:17:181
561997929,9cyclictest0-21swapper/121:17:181
549997955,17cyclictest30923-21cut20:31:550
54999794,42cyclictest15-21ksoftirqd/019:10:050
567997819,27cyclictest0-21swapper/220:47:192
567997819,27cyclictest0-21swapper/220:47:192
561997845,12cyclictest181rcu_preempt20:57:231
561997844,12cyclictest181rcu_preempt20:37:161
561997843,12cyclictest181rcu_preempt00:19:111
561997843,12cyclictest181rcu_preempt00:19:111
561997842,20cyclictest11624-21grep19:32:161
549997862,10cyclictest8184-21diskmemload00:27:150
549997862,10cyclictest8184-21diskmemload00:27:150
549997858,14cyclictest8184-21diskmemload23:55:110
561997746,11cyclictest181rcu_preempt19:06:561
549997760,11cyclictest1-21systemd20:02:090
549997760,11cyclictest1-21systemd20:02:090
549997756,15cyclictest7029-21fschecks_count20:57:170
549997731,37cyclictest0-21swapper/000:02:160
549997714,28cyclictest19676-21snmpd00:08:000
567997622,23cyclictest18687-21fschecks_time22:12:142
567997619,23cyclictest19366-21cstates23:07:112
561997634,13cyclictest0-21swapper/121:55:291
549997647,21cyclictest16021-21latency_hist19:46:560
561997546,12cyclictest181rcu_preempt20:02:181
561997546,12cyclictest181rcu_preempt20:02:181
561997531,20cyclictest0-21swapper/122:22:111
549997560,10cyclictest58-21kcompactd020:56:560
549997553,16cyclictest18136-21sh22:11:540
549997553,15cyclictest27019-21fschecks_count21:32:150
549997514,29cyclictest15-21ksoftirqd/023:26:350
561997425,21cyclictest0-21swapper/121:02:041
54999746,28cyclictest8518-21ssh21:02:070
549997453,15cyclictest297-21systemd-logind21:56:530
549997412,48cyclictest16682-21ntp_kernel_pll_19:47:220
549997412,48cyclictest16682-21ntp_kernel_pll_19:47:220
561997340,12cyclictest181rcu_preempt20:17:251
549997356,10cyclictest1721-21idleruntime-cro20:41:530
549997353,14cyclictest3030-21idleruntime-cro21:46:530
549997343,23cyclictest0-21swapper/021:21:570
549997343,23cyclictest0-21swapper/021:21:570
54999734,29cyclictest7636-21latency_hist19:21:550
549997339,27cyclictest0-21swapper/023:57:280
54999723,56cyclictest10777-21/usr/sbin/munin23:47:100
54999723,34cyclictest15-21ksoftirqd/021:13:240
54999723,26cyclictest15-21ksoftirqd/023:10:590
549997227,33cyclictest13683-21grep22:57:190
549997212,35cyclictest15-21ksoftirqd/022:26:280
3181727222,33sleep30-21swapper/318:57:223
549997155,10cyclictest9972-21ntp_kernel_pll_21:57:220
549997155,10cyclictest23961-21df21:27:140
549997155,10cyclictest23961-21df21:27:140
54999715,36cyclictest15-21ksoftirqd/000:23:450
549997151,14cyclictest536-21cyclictest21:24:130
549997146,20cyclictest536-21cyclictest20:12:290
54999713,26cyclictest27914-21cron23:21:520
549997052,13cyclictest30237-21apt-key22:31:560
549997051,13cyclictest14011-21sh21:11:250
549997050,16cyclictest536-21cyclictest22:52:300
549997050,16cyclictest536-21cyclictest22:52:300
549997047,14cyclictest6754-21sh22:46:320
549997047,14cyclictest6754-21sh22:46:320
54999703,38cyclictest15-21ksoftirqd/023:14:480
54999703,22cyclictest19189-21sh19:56:540
549997013,34cyclictest15-21ksoftirqd/000:20:200
549997013,34cyclictest15-21ksoftirqd/000:20:200
549997013,33cyclictest15-21ksoftirqd/020:36:340
549997012,34cyclictest15-21ksoftirqd/022:40:030
54999693,20cyclictest8184-21diskmemload22:12:140
54999692,25cyclictest15-21ksoftirqd/019:41:430
549996912,34cyclictest15-21ksoftirqd/021:40:340
549996912,34cyclictest15-21ksoftirqd/021:40:340
549996855,9cyclictest536-21cyclictest23:05:020
54999683,36cyclictest19946-21fschecks_time19:58:000
54999683,24cyclictest31438-21df_inode23:27:130
54999683,24cyclictest31438-21df_inode23:27:130
54999679,28cyclictest15-21ksoftirqd/023:43:160
549996751,10cyclictest4720-21latency_hist23:36:560
54999673,34cyclictest15-21ksoftirqd/020:21:130
54999673,33cyclictest15-21ksoftirqd/000:12:450
54999672,23cyclictest15-21ksoftirqd/019:36:480
549996645,15cyclictest24098-21idleruntime-cro22:21:530
54999664,37cyclictest15-21ksoftirqd/022:05:290
54999663,52cyclictest4642-21apt-get19:12:100
54999663,52cyclictest4642-21apt-get19:12:100
54999662,27cyclictest15-21ksoftirqd/023:36:580
57499652,39cyclictest0-21swapper/319:12:073
57499652,39cyclictest0-21swapper/319:12:073
54999653,32cyclictest29016-21head20:23:460
54999653,29cyclictest3545-21latency_hist20:46:570
54999644,43cyclictest23104-21fschecks_time20:07:160
54999644,24cyclictest2244-21latency_hist19:06:570
54999633,32cyclictest15-21ksoftirqd/020:51:480
54999633,32cyclictest15-21ksoftirqd/020:51:480
57499592,34cyclictest0-21swapper/319:27:073
57499563,42cyclictest24437-21taskset22:22:083
57499542,42cyclictest23511-21taskset20:07:273
57499533,36cyclictest21802-21taskset20:02:253
57499533,36cyclictest21802-21taskset20:02:253
57499532,38cyclictest12037-21chrt22:01:503
57499524,35cyclictest13586-21chrt19:37:263
57499523,38cyclictest8739-21taskset21:02:153
57499523,36cyclictest7063-21chrt22:46:563
57499523,36cyclictest7063-21chrt22:46:563
57499523,35cyclictest14193-21chrt23:52:173
57499522,35cyclictest28805-21chrt21:35:033
57499522,34cyclictest32583-21chrt23:28:103
57499522,34cyclictest32583-21chrt23:28:103
57499513,33cyclictest21929-21taskset23:11:523
57499512,37cyclictest15458-21chrt22:07:093
57499512,32cyclictest2049-21taskset20:42:103
57499503,38cyclictest6042-21chrt21:51:533
57499503,38cyclictest18442-21chrt22:12:073
57499503,37cyclictest29030-21taskset23:22:263
57499503,34cyclictest962-21chrt19:02:143
57499503,34cyclictest7191-21chrt20:57:213
57499503,30cyclictest25206-21taskset23:16:593
57499502,34cyclictest3715-21chrt19:07:333
57499502,33cyclictest28277-21chrt20:23:313
57499493,35cyclictest1714-21chrt22:37:203
57499493,32cyclictest7991-21chrt23:42:123
57499493,32cyclictest30435-21chrt22:31:573
57499493,24cyclictest0-21swapper/319:57:113
57499492,34cyclictest8530-21chrt21:55:253
574994912,22cyclictest0-21swapper/321:12:263
57499483,33cyclictest13194-21chrt22:57:013
57499483,32cyclictest21677-21chrt22:17:143
57499483,29cyclictest21897-21chrt21:22:343
57499482,34cyclictest7999-21chrt19:22:113
57499474,34cyclictest30458-21chrt21:37:263
57499474,34cyclictest30458-21chrt21:37:263
57499473,31cyclictest10939-21taskset23:47:103
57499472,36cyclictest6957-21chrt23:40:263
57499472,33cyclictest28957-21chrt00:17:093
57499472,33cyclictest28957-21chrt00:17:093
57499472,31cyclictest17588-21chrt19:51:563
57499472,31cyclictest17588-21chrt19:51:563
57499472,30cyclictest4721-21taskset00:30:593
57499472,30cyclictest4721-21taskset00:30:593
57499464,31cyclictest10515-21chrt22:52:153
57499464,31cyclictest10515-21chrt22:52:153
57499463,34cyclictest18994-21taskset21:17:523
57499463,34cyclictest18994-21taskset21:17:523
57499463,33cyclictest14476-21taskset22:05:143
57499462,32cyclictest2168-21chrt21:44:313
57499462,32cyclictest20781-21chrt00:02:303
57499462,32cyclictest1956-21taskset00:26:373
57499462,30cyclictest17107-21chrt23:57:163
57499462,24cyclictest3125-21chrt23:33:043
57499453,28cyclictest24809-21chrt21:27:333
57499453,28cyclictest24809-21chrt21:27:333
57499443,32cyclictest8932-21taskset22:49:153
57499443,32cyclictest4492-21chrt20:47:293
57499443,32cyclictest4492-21chrt20:47:293
57499443,28cyclictest24859-21chrt00:10:083
57499433,30cyclictest30934-21taskset20:31:533
57499433,27cyclictest19190-21chrt19:56:543
57499433,26cyclictest24541-21chrt23:15:503
57499432,32cyclictest28535-21taskset00:16:533
57499432,29cyclictest31879-21chrt20:32:263
57499432,27cyclictest5981-21chrt20:52:273
57499422,26cyclictest17446-21chrt23:02:493
57499413,25cyclictest26100-21chrt20:17:033
57499412,29cyclictest15915-21taskset19:46:543
574994019,13cyclictest117702chrt19:32:203
57499394,26cyclictest24514-21chrt20:12:053
57499383,26cyclictest6773-21taskset19:17:413
57499382,24cyclictest321-21taskset20:37:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional