You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 07:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Sat Jun 13, 2026 00:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19529991243,42cyclictest352-21runrttasks22:59:211
19525991242,70cyclictest4751-21cstates19:50:250
19525991242,70cyclictest4751-21cstates19:50:250
19525991152,78cyclictest20947-21apt-get20:39:320
19525991152,78cyclictest20947-21apt-get20:39:320
19525991143,72cyclictest29057-21apt-get22:49:300
19525991143,72cyclictest29057-21apt-get22:49:300
19525991073,69cyclictest15437-21modprobe00:14:300
195439910458,29cyclictest28823-21kworker/u17:1-writeback23:49:142
195439910026,28cyclictest0-21swapper/220:29:222
195439910026,28cyclictest0-21swapper/220:29:222
1952999962,31cyclictest352-21runrttasks21:03:011
1952999932,32cyclictest352-21runrttasks20:34:041
1952999932,32cyclictest352-21runrttasks20:34:041
19543999232,28cyclictest16848-21seq22:29:262
19543999232,28cyclictest16848-21seq22:29:262
19529999245,42cyclictest0-21swapper/100:15:101
1952999922,26cyclictest352-21runrttasks20:53:411
1952999912,32cyclictest352-21runrttasks19:14:011
1952999912,30cyclictest352-21runrttasks20:47:371
19525999149,29cyclictest6181-21apt-get19:54:250
19525999149,29cyclictest6181-21apt-get19:54:250
1952999904,10cyclictest0-21swapper/121:04:241
19543998970,13cyclictest29585-21sh22:49:562
19543998970,13cyclictest29585-21sh22:49:562
19543998929,21cyclictest24170-21seq20:49:192
19529998947,35cyclictest11254-21latency22:19:341
19529998947,35cyclictest11254-21latency22:19:341
19529998946,36cyclictest7041-21sh21:19:271
19525998967,15cyclictest15864-21latency_hist21:34:140
1952999884,9cyclictest0-21swapper/122:39:241
1952999884,9cyclictest0-21swapper/122:39:241
1952999884,37cyclictest8611-21apt-get23:09:221
19529998833,44cyclictest0-21swapper/120:19:231
1952999883,10cyclictest0-21swapper/118:59:231
1952599883,61cyclictest22017-21runrttasks19:04:370
19543998731,23cyclictest10535-21seq00:05:392
19529998745,35cyclictest0-21swapper/121:09:271
19529998745,35cyclictest0-21swapper/121:09:271
1952999873,43cyclictest29116-21sshd22:49:311
1952999873,43cyclictest29116-21sshd22:49:311
1952999873,36cyclictest32425-21kworker/u18:2+rpciod23:19:251
1952999872,8cyclictest0-21swapper/121:34:471
1952999872,7cyclictest0-21swapper/100:09:221
1952999872,42cyclictest12735-21sendmail-msp21:29:121
19525998763,16cyclictest20131-21latency18:59:360
19543998627,19cyclictest26813-21seq20:55:032
19543998627,19cyclictest26813-21seq20:55:032
1952999864,8cyclictest0-21swapper/119:54:251
1952999864,8cyclictest0-21swapper/119:54:251
1952999863,9cyclictest0-21swapper/122:24:201
1952999862,34cyclictest9616-21/usr/sbin/munin20:04:331
1954399854,56cyclictest15383-21apt-get00:14:282
19543998529,21cyclictest15308-21seq20:20:442
19543998521,22cyclictest15154-21runrttasks21:32:402
19529998540,38cyclictest0-21swapper/123:14:481
19529998540,38cyclictest0-21swapper/123:14:481
1952999853,39cyclictest9120-21/usr/sbin/munin00:04:221
1952999853,36cyclictest5945-21systemd19:54:131
1952999853,36cyclictest5945-21systemd19:54:131
1952999852,8cyclictest0-21swapper/119:44:281
1952999852,8cyclictest0-21swapper/119:44:281
19543998429,24cyclictest26426-21seq19:17:212
19543998422,28cyclictest6084-21latency_hist19:54:152
19543998422,28cyclictest6084-21latency_hist19:54:152
1952999845,73cyclictest0-21swapper/119:59:261
1952999845,73cyclictest0-21swapper/119:59:261
19529998442,35cyclictest0-21swapper/119:39:461
1952999843,40cyclictest352-21runrttasks23:44:411
1952999843,39cyclictest28216-21gpgv19:24:121
1952999843,39cyclictest28216-21gpgv19:24:121
1952999843,34cyclictest2908-21grep23:54:141
1952999843,34cyclictest2908-21grep23:54:141
1952999842,35cyclictest27708-21/usr/sbin/munin23:39:491
19525998418,32cyclictest0-21swapper/020:24:280
19543998327,23cyclictest17049-21seq20:27:282
1952999839,66cyclictest0-21swapper/121:40:541
19529998340,36cyclictest14455-21sh21:30:431
1952999833,7cyclictest0-21swapper/122:54:271
1952999833,40cyclictest24871-21ssh23:35:161
1952999833,40cyclictest24871-21ssh23:35:161
1952999833,39cyclictest20321-21sed22:34:381
1952999833,35cyclictest20929-21/usr/sbin/munin20:40:251
1952999833,35cyclictest20929-21/usr/sbin/munin20:40:251
1952999833,33cyclictest29989-21taskset21:56:341
1952999833,33cyclictest29989-21taskset21:56:341
1952999833,33cyclictest21496-21munin-node19:04:181
1952999833,33cyclictest14105-21latency_hist20:19:141
1952999833,31cyclictest31916-21/usr/sbin/munin19:34:351
1952999833,31cyclictest31916-21/usr/sbin/munin19:34:351
1952999832,9cyclictest0-21swapper/120:09:251
1952999832,33cyclictest16973-21seq20:26:311
19543998231,19cyclictest29404-21seq21:02:082
19529998254,20cyclictest17784-21kworker/u18:0+nfsiod23:26:321
1952999824,32cyclictest20592-21/usr/sbin/munin23:29:471
1952999823,72cyclictest6178-21ssh23:59:271
1952999823,34cyclictest3804-21runrttasks23:54:451
1952999823,33cyclictest8572-21/usr/sbin/munin22:14:451
1952999823,32cyclictest277-21dbus-daemon19:34:131
1952999823,32cyclictest277-21dbus-daemon19:34:131
1952999822,33cyclictest20449-21apt-get20:39:111
1952999822,32cyclictest4019-21rm21:14:201
19525998259,17cyclictest21934-21idleruntime-cro21:44:120
19525998258,18cyclictest14103-21latency_hist20:19:140
1952999815,69cyclictest0-21swapper/121:59:221
1952999814,8cyclictest0-21swapper/120:54:251
1952999814,8cyclictest0-21swapper/120:54:251
1952999813,35cyclictest16796-21/usr/sbin/munin22:29:211
1952999813,35cyclictest16796-21/usr/sbin/munin22:29:211
1952999813,33cyclictest25935-21cstates22:44:291
1952999813,32cyclictest2304-21taskset22:04:371
1952999813,32cyclictest2304-21taskset22:04:371
1952999813,30cyclictest29455-21/usr/sbin/munin19:25:461
1952999813,30cyclictest29455-21/usr/sbin/munin19:25:461
1952999812,32cyclictest21388-21/usr/sbin/munin00:24:261
19525998157,16cyclictest29939-21sh21:03:380
19525998139,27cyclictest16816-21apt-get22:29:250
19525998139,27cyclictest16816-21apt-get22:29:250
1952599813,22cyclictest28690-21apt-get19:24:260
1952599813,22cyclictest28690-21apt-get19:24:260
19543998058,14cyclictest30414-21cstates19:29:272
19543998058,14cyclictest30414-21cstates19:29:272
19543998038,9cyclictest0-21swapper/223:14:152
19543998029,20cyclictest28519-21seq23:42:052
1952999804,9cyclictest0-21swapper/121:49:251
1952999804,32cyclictest18650-21fschecks_time00:19:331
1952999803,35cyclictest5090-21fschecks_count22:09:331
1952999803,33cyclictest24419-21taskset21:47:501
1952999803,31cyclictest7773-21chrt23:07:491
19525998052,18cyclictest28493-21latency_hist22:49:150
19525998050,21cyclictest0-21swapper/023:39:140
19525998050,21cyclictest0-21swapper/023:39:140
191222806,55sleep10-21swapper/118:55:461
19543997961,9cyclictest0-21swapper/220:39:122
19543997958,15cyclictest544-21cron23:19:112
19543997958,15cyclictest544-21cron23:19:112
19543997923,24cyclictest5581-21mailstats22:09:452
19543997922,26cyclictest8651-21runrttasks00:03:452
19525997961,11cyclictest14132-21kworker/u18:2-writeback00:24:390
19525997959,14cyclictest19516-21cyclictest23:56:080
19525997958,12cyclictest23084-21ssh22:39:340
19525997958,12cyclictest23084-21ssh22:39:340
19543997845,26cyclictest9683-21latency_hist21:24:152
19543997832,24cyclictest1791-21seq22:04:222
19543997832,24cyclictest1791-21seq22:04:222
19543997829,25cyclictest27485-21runrttasks21:52:552
1952999783,30cyclictest25226-21/usr/sbin/munin19:14:221
19525997858,14cyclictest21052-21idleruntime-cro00:24:110
1952599782,26cyclictest18834-21idleruntime-cro20:34:110
1952599782,26cyclictest18834-21idleruntime-cro20:34:110
19543997730,23cyclictest3876-21seq23:00:182
19525997739,28cyclictest0-21swapper/022:04:430
19525997739,28cyclictest0-21swapper/022:04:430
1955099763,50cyclictest0-21swapper/322:49:383
1955099763,50cyclictest0-21swapper/322:49:383
19550997616,37cyclictest0-21swapper/319:50:253
19550997616,37cyclictest0-21swapper/319:50:253
19543997638,8cyclictest181rcu_preempt22:49:152
19543997622,27cyclictest22681-21runrttasks23:32:322
19525997660,10cyclictest21064-21ssh22:35:130
19525997654,16cyclictest2945-21latency_hist23:54:140
19525997654,16cyclictest2945-21latency_hist23:54:140
19525997635,12cyclictest0-21swapper/023:48:310
1945427610,41sleep019456-21munin-run18:59:150
19543997554,14cyclictest16225-21/usr/sbin/munin21:35:012
19543997529,19cyclictest23128-21seq19:06:352
19543997435,19cyclictest32448-21if_eth022:54:352
19543997427,23cyclictest5252-21runrttasks21:15:092
19543997425,24cyclictest0-21swapper/222:15:572
19525997443,23cyclictest15067-21latency23:19:360
19525997442,25cyclictest0-21swapper/023:34:110
193962739,40sleep219414-21munin-run18:59:132
19543997251,8cyclictest181rcu_preempt21:39:162
19543997247,16cyclictest25545-21ssh22:44:142
19543997247,16cyclictest25545-21ssh22:44:142
19543997246,19cyclictest6378-21swap23:04:452
19543997246,19cyclictest31483-21fschecks_count21:59:332
19543997239,26cyclictest10026-21ssh21:24:272
19543997235,28cyclictest0-21swapper/200:09:242
19543997234,21cyclictest8019-21fschecks_time19:59:332
19543997234,21cyclictest8019-21fschecks_time19:59:332
19525997253,12cyclictest19516-21cyclictest00:06:100
19525997251,12cyclictest8804-21ssh21:21:450
19525997234,29cyclictest0-21swapper/022:56:010
19525997229,33cyclictest0-21swapper/000:14:150
1955099713,43cyclictest0-21swapper/319:04:273
19543997153,11cyclictest23912-21fschecks_time23:34:332
19543997153,11cyclictest23912-21fschecks_time23:34:332
19543997152,13cyclictest21624-21fschecks_time00:24:322
19543997149,15cyclictest17440-21latency_hist23:24:142
19543997148,17cyclictest6757-21kworker/u17:1-rpciod00:19:222
19543997130,36cyclictest0-21swapper/220:14:222
19543997123,33cyclictest32638-21awk23:49:252
19543997123,33cyclictest32638-21awk23:49:252
19525997153,12cyclictest14322-21ssh23:19:130
19525997153,12cyclictest14322-21ssh23:19:130
1952599713,29cyclictest28600-21idleruntime21:54:340
1952599713,29cyclictest28600-21idleruntime21:54:340
19543997044,19cyclictest29593-21timerwakeupswit19:25:482
19543997044,19cyclictest29593-21timerwakeupswit19:25:482
19525997056,8cyclictest2895-21ssh22:59:320
19525997053,12cyclictest0-21swapper/020:54:130
19525997051,10cyclictest0-21swapper/021:18:310
1952599703,53cyclictest10319-21fschecks_time21:24:350
19543996952,12cyclictest22649-21sh21:44:382
19543996938,26cyclictest0-21swapper/220:04:232
19543996934,25cyclictest3745-21/usr/sbin/munin23:54:432
19543996932,27cyclictest21058-21apt19:04:122
19525996950,13cyclictest30899-21proc_pri19:29:420
19525996950,13cyclictest30899-21proc_pri19:29:420
19525996948,14cyclictest0-21swapper/000:04:110
19525996945,18cyclictest7647-21sh22:14:140
1952599693,31cyclictest10734-21modprobe20:09:110
19525996929,31cyclictest9329-21wc23:09:440
1952599692,49cyclictest27160-21diskmemload22:19:320
1952599692,49cyclictest27160-21diskmemload22:19:320
19543996842,18cyclictest11550-21ntp_kernel_pll_20:09:382
19543996838,20cyclictest26848-21apt-get19:19:272
19543996838,20cyclictest26848-21apt-get19:19:272
19543996834,24cyclictest814-21modprobe19:39:112
19543996834,24cyclictest814-21modprobe19:39:112
19525996846,16cyclictest15670-21latency_hist20:24:140
1952599683,27cyclictest22825-21fschecks_time20:44:350
1952599682,33cyclictest18530-21ssh23:24:470
19525996821,21cyclictest181rcu_preempt20:39:110
19525996817,26cyclictest27160-21diskmemload22:19:150
1955099674,41cyclictest0-21swapper/300:14:293
19543996738,21cyclictest0-21swapper/221:06:122
19543996732,27cyclictest16732-21ls22:29:152
19543996730,27cyclictest11201-21runrttasks22:19:332
19543996730,27cyclictest11201-21runrttasks22:19:332
19525996743,16cyclictest24217-21processes19:09:430
19525996741,20cyclictest2069-21ssh21:09:490
19525996741,20cyclictest2069-21ssh21:09:490
19543996635,23cyclictest0-21swapper/221:09:402
19543996635,23cyclictest0-21swapper/221:09:402
19543996633,26cyclictest21296-21cut20:40:412
19543996633,26cyclictest21296-21cut20:40:412
19543996633,26cyclictest20926-21grep22:34:542
19543996632,26cyclictest4323-21latency_hist19:49:152
19543996632,26cyclictest4323-21latency_hist19:49:152
19543996630,24cyclictest22550-21/usr/sbin/munin20:44:232
19543996629,26cyclictest2441-21basename19:44:112
19525996634,25cyclictest0-21swapper/021:34:260
19525996634,22cyclictest0-21swapper/022:24:360
185432668,33sleep30-21swapper/318:54:463
19543996532,25cyclictest23632-21apt19:09:212
19543996529,26cyclictest28268-21ssh21:54:222
19543996529,26cyclictest28268-21ssh21:54:222
19543996529,25cyclictest7405-21seq19:57:512
19543996529,25cyclictest7405-21seq19:57:512
19525996547,12cyclictest26562-21threads20:54:470
19525996547,12cyclictest26562-21threads20:54:470
1952599653,31cyclictest1558-21netstat19:39:380
1952599653,29cyclictest11014-21/usr/sbin/munin20:09:320
19525996522,26cyclictest32156-21sh21:59:470
19543996432,23cyclictest17577-21sort23:24:222
19525996445,14cyclictest25011-21latency_hist21:49:130
19525996440,18cyclictest28133-21ssh23:41:000
1952599643,46cyclictest25341-21cstates19:14:290
1952599643,30cyclictest492-21kworker/u17:1+ext4-rsv-conversion19:39:140
1952599643,30cyclictest492-21kworker/u17:1+ext4-rsv-conversion19:39:140
1955099634,43cyclictest12837-21taskset20:14:313
19525996346,11cyclictest8294-21sendmail23:09:110
19525996342,15cyclictest28583-21latency_hist19:24:160
19525996342,15cyclictest28583-21latency_hist19:24:160
1952599633,22cyclictest30701-21fschecks_count21:04:330
1952599632,22cyclictest4178-21apt-key19:49:140
1952599632,22cyclictest4178-21apt-key19:49:140
1955099622,45cyclictest16323-21chrt21:34:343
19525996244,13cyclictest25554-21fschecks_time21:49:330
1952599623,40cyclictest7768-21/usr/sbin/munin19:59:370
1952599623,40cyclictest7768-21/usr/sbin/munin19:59:370
1955099613,34cyclictest0-21swapper/323:49:113
1955099573,39cyclictest17663-21chrt22:29:463
1955099573,39cyclictest17663-21chrt22:29:463
1955099573,38cyclictest13194-21chrt00:09:483
1955099552,41cyclictest2896-21chrt19:44:343
1955099552,41cyclictest2896-21chrt19:44:343
19550995513,22cyclictest0-21swapper/320:39:323
19550995513,22cyclictest0-21swapper/320:39:323
1955099543,37cyclictest8517-21taskset21:20:563
1955099543,37cyclictest20005-21taskset18:59:333
1955099543,36cyclictest24514-21taskset19:09:463
1955099542,36cyclictest19305-21taskset20:34:323
1955099524,34cyclictest12526-21chrt20:14:153
1955099523,39cyclictest20251-21chrt21:39:493
1955099522,39cyclictest13670-21taskset21:29:433
1955099522,37cyclictest11393-21taskset00:08:013
1955099513,38cyclictest16185-21taskset22:28:243
1955099513,37cyclictest31767-21chrt21:59:383
1955099513,36cyclictest5789-21chrt22:09:483
1955099513,36cyclictest4795-21chrt23:56:183
1955099513,36cyclictest4195-21chrt21:14:273
1955099513,35cyclictest15774-21chrt20:24:153
1955099513,34cyclictest3144-21chrt21:12:373
1955099513,34cyclictest3144-21chrt21:12:373
1955099512,34cyclictest16483-21chrt20:24:443
1955099509,28cyclictest0-21swapper/319:24:263
1955099509,28cyclictest0-21swapper/319:24:263
1955099504,34cyclictest11452-21chrt22:19:413
1955099504,34cyclictest11452-21chrt22:19:413
19550995035,8cyclictest451rcuc/319:43:493
1955099503,35cyclictest9973-21chrt22:17:463
1955099503,35cyclictest27495-21chrt22:46:273
1955099503,35cyclictest25589-21chrt19:14:363
1955099503,33cyclictest26748-21chrt21:50:513
1955099503,33cyclictest24083-21chrt20:49:143
1955099503,33cyclictest18503-21chrt00:19:283
1955099502,35cyclictest19575-21chrt23:27:163
1955099493,36cyclictest1819-21chrt23:51:203
1955099493,36cyclictest1819-21chrt23:51:203
1955099493,33cyclictest8469-21taskset19:59:453
1955099493,33cyclictest8469-21taskset19:59:453
1955099493,32cyclictest6917-21chrt19:54:463
1955099493,32cyclictest6917-21chrt19:54:463
1955099493,32cyclictest23841-21taskset00:28:403
1955099493,26cyclictest1044-21chrt19:39:153
1955099492,35cyclictest31041-21taskset21:04:423
1955099492,35cyclictest14472-21chrt23:19:143
1955099492,35cyclictest14472-21chrt23:19:143
1955099492,34cyclictest21720-21chrt22:37:033
1955099492,33cyclictest714-21chrt22:54:503
19550994833,8cyclictest451rcuc/319:21:493
19550994833,8cyclictest451rcuc/319:21:493
1955099483,36cyclictest30914-21chrt21:59:103
1955099483,36cyclictest30914-21chrt21:59:103
1955099483,34cyclictest29694-21chrt21:02:563
1955099483,33cyclictest5991-21chrt23:04:353
1955099483,33cyclictest4284-21chrt23:01:273
1955099482,32cyclictest32165-21taskset19:34:333
1955099482,32cyclictest16626-21chrt23:22:263
1955099482,29cyclictest29507-21chrt23:44:143
1955099473,31cyclictest22491-21chrt21:44:353
1955099472,33cyclictest25454-21chrt23:36:533
1955099472,33cyclictest25454-21chrt23:36:533
1955099472,32cyclictest12817-21taskset21:29:153
1955099463,33cyclictest27298-21taskset20:59:153
1955099463,33cyclictest27298-21taskset20:59:153
1955099463,31cyclictest4677-21chrt22:09:153
1955099463,31cyclictest4677-21chrt22:09:153
1955099453,33cyclictest25391-21chrt22:43:563
1955099453,33cyclictest25391-21chrt22:43:563
1955099453,33cyclictest18258-21chrt20:29:463
1955099453,33cyclictest18258-21chrt20:29:463
1955099452,31cyclictest7340-21chrt00:00:033
1955099443,32cyclictest10677-21chrt20:08:323
1955099442,28cyclictest21585-21chrt23:29:483
1955099433,32cyclictest30421-21chrt19:29:273
1955099433,32cyclictest30421-21chrt19:29:273
1955099423,31cyclictest8794-21chrt23:09:293
1955099373,25cyclictest24672-21taskset20:49:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional