You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 16:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Mon Sep 07, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1794021117,18sleep00-21swapper/007:07:400
16941210710,69sleep116947-21/usr/sbin/munin07:05:081
182509910211,36cyclictest0-21swapper/011:04:570
18255999833,31cyclictest23192-21seq10:50:031
18262999453,18cyclictest181rcu_preempt10:07:522
18255999474,13cyclictest16975-21runrttasks10:39:541
18262999356,13cyclictest181rcu_preempt07:14:482
18262999356,13cyclictest181rcu_preempt07:14:482
18250999169,14cyclictest4227-21latency_hist08:04:440
1825099912,78cyclictest7778-21kworker/u19:0+ext4-rsv-conversion12:10:350
18255998927,25cyclictest200-21systemd-journal09:44:581
18255998927,25cyclictest200-21systemd-journal09:44:581
18250998926,54cyclictest1370-21apt-get10:14:570
1826299886,60cyclictest9479-21ssh11:20:102
1825099882,60cyclictest9418-21sh11:20:000
18262998770,8cyclictest0-21swapper/209:45:032
18262998770,8cyclictest0-21swapper/209:45:032
18262998762,17cyclictest326-21systemd-logind10:04:432
1826299873,34cyclictest23730-21latency_hist07:24:442
1826299873,34cyclictest23730-21latency_hist07:24:442
18255998737,34cyclictest5659-21apt-get09:29:531
1826299867,57cyclictest23428-21apt-get12:35:012
18262998644,15cyclictest181rcu_preempt08:20:042
18262998644,15cyclictest181rcu_preempt08:20:042
1826299863,62cyclictest17162-21ssh10:40:212
18262998554,12cyclictest181rcu_preempt10:35:082
18262998549,10cyclictest181rcu_preempt12:07:022
18262998544,13cyclictest32508-21ssh11:05:002
18255998531,24cyclictest11121-21taskset10:30:081
18250998565,15cyclictest12013-21vmstat12:15:210
18250998565,15cyclictest12013-21vmstat12:15:210
18262998429,13cyclictest181rcu_preempt11:00:202
1825599846,68cyclictest23505-21ssh12:35:041
18255998458,19cyclictest3571-21ssh12:02:581
18255998432,39cyclictest0-21swapper/111:39:541
18255998432,39cyclictest0-21swapper/111:39:541
18255998428,25cyclictest32130-21taskset10:11:071
18250998465,13cyclictest29494-21sh11:51:580
18255998327,20cyclictest0-21swapper/109:53:131
18262998252,12cyclictest181rcu_preempt09:10:542
18255998256,19cyclictest19248-21timerandwakeup07:10:191
18255998256,19cyclictest19248-21timerandwakeup07:10:191
18255998222,27cyclictest13012-21ntpq11:25:101
18255998222,27cyclictest13012-21ntpq11:25:101
18250998260,14cyclictest1128-21idleruntime-cro11:59:420
18262998146,13cyclictest181rcu_preempt09:00:382
1826299812,33cyclictest20127-21apt-get12:29:522
18255998129,36cyclictest7002-21vmstat08:10:211
18255998127,22cyclictest0-21swapper/111:38:401
18255998126,22cyclictest3986-21taskset10:19:231
18255998123,23cyclictest0-21swapper/109:55:571
18250998159,17cyclictest20549-21latency10:45:080
18250998159,17cyclictest20549-21latency10:45:080
18262998047,18cyclictest181rcu_preempt11:54:532
18262998037,11cyclictest181rcu_preempt11:53:152
18262998015,38cyclictest2277-21ssh12:00:172
18250998057,16cyclictest20936-21fschecks_count07:15:380
18250998057,16cyclictest20936-21fschecks_count07:15:380
18262997957,13cyclictest21333-21ssh10:45:452
18262997957,13cyclictest21333-21ssh10:45:452
18262997957,12cyclictest25572-21latency_hist09:09:462
18262997957,12cyclictest25572-21latency_hist09:09:462
18262997948,9cyclictest181rcu_preempt08:26:332
18262997948,9cyclictest181rcu_preempt08:26:332
18262997934,13cyclictest181rcu_preempt07:55:152
18262997934,13cyclictest181rcu_preempt07:55:152
18262997927,10cyclictest0-21swapper/208:34:542
18255997957,15cyclictest8979-21systemctl08:19:421
18255997938,32cyclictest0-21swapper/108:19:571
18255997938,32cyclictest0-21swapper/108:19:571
18255997929,18cyclictest27470-21taskset09:12:041
18255997927,25cyclictest343-21runrttasks12:32:161
18255997927,20cyclictest343-21runrttasks11:15:211
18255997927,20cyclictest343-21runrttasks11:15:211
18255997926,26cyclictest14557-21taskset09:44:301
18255997920,27cyclictest0-21swapper/110:46:171
18255997920,27cyclictest0-21swapper/110:46:171
1825099799,60cyclictest16995-21apt-get10:40:190
1825099793,64cyclictest23474-21ssh12:35:000
18262997857,15cyclictest662-21cron08:59:422
18262997857,15cyclictest524-21in:imuxsock11:49:422
18262997854,15cyclictest6927-21timerandwakeup11:15:182
18262997854,15cyclictest6927-21timerandwakeup11:15:182
18262997848,20cyclictest32209-21fschecks_count07:50:052
18262997847,8cyclictest181rcu_preempt09:15:342
18262997843,9cyclictest181rcu_preempt07:10:052
18262997843,9cyclictest181rcu_preempt07:10:052
18262997814,35cyclictest39-21ksoftirqd/210:29:532
1825599784,38cyclictest343-21runrttasks12:19:511
1825599783,43cyclictest31580-21sendmail09:19:421
18250997857,15cyclictest524-21in:imuxsock08:59:430
18262997755,14cyclictest200-21systemd-journal09:33:582
18262997748,8cyclictest181rcu_preempt09:51:352
18262997747,13cyclictest181rcu_preempt12:28:312
18262997736,16cyclictest181rcu_preempt11:12:302
18262997722,12cyclictest181rcu_preempt12:15:062
18262997722,12cyclictest181rcu_preempt12:15:062
18255997758,12cyclictest13784-21ssh12:19:451
18255997758,12cyclictest13784-21ssh12:19:451
18255997737,34cyclictest0-21swapper/109:34:581
18255997733,38cyclictest0-21swapper/111:20:081
18255997730,36cyclictest25379-21diskmemload09:24:591
18255997730,36cyclictest25379-21diskmemload09:24:591
18255997726,19cyclictest29182-21taskset10:06:301
18255997724,25cyclictest22329-21/usr/sbin/munin07:20:201
18255997724,25cyclictest22329-21/usr/sbin/munin07:20:201
18255997721,28cyclictest20256-21munin-node10:54:511
1809227724,35sleep30-21swapper/307:09:423
1826299763,57cyclictest30742-21ssh10:09:562
18255997654,13cyclictest5388-21/usr/sbin/munin08:05:191
18255997627,19cyclictest734-21taskset11:58:411
18255997627,19cyclictest29366-21taskset11:51:361
18255997626,21cyclictest25379-21diskmemload10:37:001
18250997661,10cyclictest14314-21ssh11:27:240
18250997661,10cyclictest14314-21ssh11:27:240
18250997655,17cyclictest22683-21kworker/u19:2-events_unbound11:04:410
18262997547,10cyclictest181rcu_preempt08:00:102
18262997535,13cyclictest181rcu_preempt11:40:342
18262997535,13cyclictest181rcu_preempt11:40:342
18262997530,16cyclictest181rcu_preempt12:21:352
18262997522,12cyclictest181rcu_preempt11:25:032
18262997522,12cyclictest181rcu_preempt11:25:032
18262997516,28cyclictest0-21swapper/209:40:062
18255997554,14cyclictest18240-21cyclictest11:01:031
18250997557,12cyclictest12676-21fschecks_count08:30:040
18250997556,12cyclictest5789-21rm09:29:590
18250997556,11cyclictest11587-21ssh09:39:440
18250997519,34cyclictest0-21swapper/008:39:450
18262997447,12cyclictest10797-21ssh09:37:492
18262997442,8cyclictest181rcu_preempt07:35:102
18262997441,11cyclictest181rcu_preempt11:33:542
1825599747,55cyclictest8091-21ssh12:10:331
18255997438,31cyclictest0-21swapper/107:59:561
18255997418,24cyclictest5802-21ssh10:20:561
18255997414,24cyclictest1896-21sh11:07:401
18250997459,10cyclictest27412-21ssh11:49:370
18250997454,14cyclictest326-21systemd-logind11:19:420
18250997454,14cyclictest326-21systemd-logind11:19:420
18250997453,14cyclictest31749-21cut07:49:450
18262997345,12cyclictest181rcu_preempt09:25:182
18262997345,12cyclictest181rcu_preempt09:25:182
18262997340,10cyclictest181rcu_preempt10:25:032
18262997340,10cyclictest181rcu_preempt10:25:032
18262997336,7cyclictest181rcu_preempt10:50:122
18262997336,11cyclictest181rcu_preempt10:15:022
18262997334,16cyclictest181rcu_preempt11:35:112
18262997333,17cyclictest181rcu_preempt12:13:362
1826299732,29cyclictest39-21ksoftirqd/208:08:562
18255997334,29cyclictest7533-21apt-get10:24:531
18255997334,29cyclictest7533-21apt-get10:24:531
1825599732,49cyclictest28665-21latency_hist07:39:451
18255997320,21cyclictest7090-21sh12:09:121
18255997318,28cyclictest0-21swapper/107:58:461
18255997318,28cyclictest0-21swapper/107:58:461
18255997312,25cyclictest0-21swapper/111:30:561
18250997345,20cyclictest18299-21sort09:49:540
1825099733,52cyclictest11115-21memory10:30:080
18250997320,17cyclictest25379-21diskmemload10:20:110
18262997246,10cyclictest181rcu_preempt08:40:342
18262997246,10cyclictest181rcu_preempt08:40:342
18262997236,13cyclictest181rcu_preempt09:24:462
18262997228,12cyclictest181rcu_preempt07:25:082
18262997211,37cyclictest0-21swapper/209:54:582
18255997251,12cyclictest2993-21apt-get11:09:561
18255997234,30cyclictest16650-21mailstats08:40:331
18255997234,30cyclictest16650-21mailstats08:40:331
18255997234,26cyclictest0-21swapper/109:04:531
18255997234,26cyclictest0-21swapper/109:04:531
18255997231,31cyclictest32102-21ssh09:20:011
18255997226,36cyclictest11910-21sshd08:25:581
18255997226,36cyclictest11910-21sshd08:25:581
1825099726,31cyclictest0-21swapper/012:29:550
18250997224,28cyclictest15-21ksoftirqd/008:16:000
1826299716,40cyclictest39-21ksoftirqd/208:30:182
18262997153,10cyclictest0-21swapper/210:59:422
18262997147,8cyclictest181rcu_preempt07:30:152
18262997142,12cyclictest181rcu_preempt08:49:422
18262997142,12cyclictest181rcu_preempt08:49:422
18255997151,14cyclictest18240-21cyclictest07:40:111
18255997149,16cyclictest524-21in:imuxsock07:49:431
18255997132,26cyclictest19376-21/usr/sbin/munin08:50:061
18255997120,25cyclictest0-21swapper/112:27:531
1825099718,31cyclictest15032-21seq10:36:010
18250997155,11cyclictest18240-21cyclictest12:00:110
18250997151,14cyclictest19886-21apt-key07:14:450
18250997151,14cyclictest19886-21apt-key07:14:450
18262997041,13cyclictest181rcu_preempt08:14:452
18255997025,18cyclictest25379-21diskmemload11:46:151
18250997051,12cyclictest909-21idleruntime-cro07:54:430
18250997049,15cyclictest526-21rs:main0
1825099702,27cyclictest1274-21latency_hist10:14:460
1826299694,43cyclictest7981-21grep08:15:082
18262996937,14cyclictest181rcu_preempt10:21:422
18262996935,16cyclictest181rcu_preempt07:40:032
18262996926,11cyclictest181rcu_preempt07:45:052
1825599693,38cyclictest15738-21latency_hist08:39:461
18255996932,26cyclictest24552-21ssh09:59:591
18255996932,25cyclictest27060-21ls07:34:451
18255996929,31cyclictest13827-21modprobe08:34:421
18250996954,9cyclictest488-21ssh09:20:200
18250996951,12cyclictest1-21systemd07:27:120
18250996947,16cyclictest7474-21latency_hist08:14:450
1825099693,32cyclictest3860-21awk11:10:190
18250996914,32cyclictest15-21ksoftirqd/012:08:200
1825599683,43cyclictest18586-21sshd08:45:411
1825599683,43cyclictest18586-21sshd08:45:411
1825599682,51cyclictest20720-21modprobe07:15:031
1825599682,51cyclictest20720-21modprobe07:15:031
18250996855,9cyclictest18240-21cyclictest12:28:060
18250996851,11cyclictest16144-21fschecks_count08:40:200
18250996851,11cyclictest16144-21fschecks_count08:40:200
18250996850,12cyclictest24892-21ssh10:52:050
18250996818,33cyclictest4764-21kworker/u17:2+nfsiod09:29:460
18250996818,33cyclictest4764-21kworker/u17:2+nfsiod09:29:460
18250996816,43cyclictest19375-21cat11:35:150
18250996814,28cyclictest15-21ksoftirqd/011:32:240
18250996812,27cyclictest15-21ksoftirqd/007:42:440
18262996728,11cyclictest181rcu_preempt08:50:072
18255996732,25cyclictest902-21munin-run07:54:431
18255996731,26cyclictest22764-21/usr/sbin/munin09:00:071
18250996752,10cyclictest26751-21ntp_kernel_err10:55:090
18250996749,11cyclictest343-21runrttasks11:44:450
18250996749,11cyclictest343-21runrttasks11:44:450
1825099673,34cyclictest14807-21switchtime12:20:170
1825099672,26cyclictest28687-21latency_hist09:14:460
18250996714,27cyclictest15-21ksoftirqd/009:44:330
18255996636,20cyclictest23952-21apt-get07:24:541
18255996624,30cyclictest21434-21/usr/sbin/munin08:55:151
1825099669,30cyclictest15-21ksoftirqd/008:21:110
1825099669,30cyclictest15-21ksoftirqd/008:21:110
1825099667,46cyclictest15195-21apt-get09:44:570
1825099667,46cyclictest15195-21apt-get09:44:570
1825099666,27cyclictest28144-21/usr/sbin/munin10:05:090
18250996649,11cyclictest526-21rs:main0
18250996649,11cyclictest526-21rs:main0
1825099661,28cyclictest22002-21sh09:55:100
1678026612,25sleep20-21swapper/207:04:572
18250996539,7cyclictest181rcu_preempt07:34:510
1825099652,27cyclictest26008-21ssh10:01:170
1825099649,31cyclictest15-21ksoftirqd/008:26:500
1825099649,31cyclictest15-21ksoftirqd/008:26:500
18250996442,11cyclictest0-21swapper/008:45:040
18250996442,11cyclictest0-21swapper/008:45:040
18250996427,30cyclictest0-21swapper/007:24:420
18250996427,30cyclictest0-21swapper/007:24:420
18250996422,19cyclictest211rcuc/008:05:160
18250996313,12cyclictest0-21swapper/009:16:550
18250996238,13cyclictest0-21swapper/007:54:590
18250996238,13cyclictest0-21swapper/007:54:590
1825099613,24cyclictest359-21snmpd08:50:200
1825099611,26cyclictest15-21ksoftirqd/007:33:400
1825099602,29cyclictest15-21ksoftirqd/009:05:020
1825099602,29cyclictest15-21ksoftirqd/009:05:020
1826799586,41cyclictest8132-21chrt12:10:403
1826799573,44cyclictest12130-21taskset11:24:423
1826799563,42cyclictest19366-21taskset12:28:313
1826799562,40cyclictest2522-21chrt10:15:263
1826799553,39cyclictest17195-21taskset10:40:253
1826799553,38cyclictest8449-21taskset10:25:183
1826799553,38cyclictest8449-21taskset10:25:183
1826799553,38cyclictest25041-21chrt10:00:103
1826799553,37cyclictest30365-21chrt11:54:203
1826799543,38cyclictest3369-21taskset11:10:073
1826799543,35cyclictest7619-21taskset09:32:353
1826799534,38cyclictest7391-21taskset12:09:433
1826799533,39cyclictest25725-21taskset11:45:203
1826799533,37cyclictest12919-21chrt08:30:083
1826799533,37cyclictest1155-21taskset07:54:533
1826799533,37cyclictest1155-21taskset07:54:533
1826799533,36cyclictest19868-21chrt07:14:443
1826799533,36cyclictest19868-21chrt07:14:443
1826799533,35cyclictest7422-21taskset10:24:453
1826799533,35cyclictest18871-21chrt11:35:033
1826799524,35cyclictest3228-21chrt12:02:013
1826799523,36cyclictest22632-21chrt07:20:073
1826799523,36cyclictest22632-21chrt07:20:073
1826799523,36cyclictest21218-21chrt08:55:073
1826799523,35cyclictest20527-21chrt12:30:053
1826799523,35cyclictest16685-21chrt08:40:333
1826799523,35cyclictest16685-21chrt08:40:333
1826799523,34cyclictest25724-21chrt09:09:573
1826799523,34cyclictest1033-21chrt11:05:213
1826799522,41cyclictest21017-21chrt09:54:353
1826799514,37cyclictest31314-21chrt11:55:093
1826799513,39cyclictest12024-21taskset10:31:103
1826799513,35cyclictest11560-21chrt12:15:143
1826799513,35cyclictest11560-21chrt12:15:143
1826799513,34cyclictest945-21chrt09:21:163
1826799512,40cyclictest517-21chrt07:50:203
1826799504,35cyclictest14177-21taskset12:20:003
1826799503,35cyclictest15074-21chrt11:29:283
1826799503,35cyclictest15074-21chrt11:29:283
1826799503,34cyclictest24199-21chrt09:05:033
1826799503,34cyclictest24199-21chrt09:05:033
1826799503,34cyclictest23387-21chrt11:42:003
1826799503,34cyclictest23387-21chrt11:42:003
1826799503,34cyclictest15126-21chrt10:36:163
1826799503,32cyclictest31512-21chrt11:03:193
1826799503,32cyclictest1151-21chrt10:14:453
1826799502,36cyclictest28963-21chrt10:59:423
1826799502,30cyclictest7527-21chrt08:14:453
1826799502,30cyclictest29300-21chrt09:15:083
18267995012,22cyclictest0-21swapper/312:35:043
1826799494,31cyclictest9667-21chrt09:35:183
1826799493,35cyclictest16400-21chrt11:30:173
1826799492,34cyclictest21331-21chrt07:15:493
1826799492,34cyclictest21331-21chrt07:15:493
1826799492,32cyclictest10602-21chrt08:24:423
1826799492,32cyclictest10602-21chrt08:24:423
1826799483,32cyclictest27912-21chrt10:05:043
1826799483,32cyclictest24209-21chrt07:25:053
1826799483,32cyclictest22163-21chrt10:47:593
1826799483,32cyclictest22163-21chrt10:47:593
1826799482,35cyclictest7547-21chrt11:16:233
1826799482,35cyclictest7547-21chrt11:16:233
1826799482,32cyclictest22973-21chrt09:00:133
18267994811,27cyclictest14351-21taskset09:43:573
1826799475,29cyclictest24399-21chrt09:59:463
1826799473,31cyclictest8100-21taskset08:15:103
1826799473,29cyclictest28534-21chrt07:39:433
1826799472,36cyclictest30195-21chrt07:44:453
1826799472,35cyclictest24866-21taskset10:52:003
1826799463,33cyclictest15018-21chrt08:35:193
1826799454,32cyclictest19018-21chrt08:49:443
1826799454,32cyclictest19018-21chrt08:49:443
1826799454,30cyclictest15811-21taskset09:45:233
1826799454,30cyclictest15811-21taskset09:45:233
1826799443,26cyclictest26958-21taskset07:34:443
18267994411,23cyclictest31218-21chrt07:45:173
1826799435,27cyclictest20174-21taskset08:50:253
1826799433,30cyclictest3207-21taskset09:25:133
1826799433,30cyclictest3207-21taskset09:25:133
18267994312,21cyclictest4973-21chrt08:05:143
1826799409,21cyclictest3806-21chrt08:00:323
1826799393,27cyclictest11428-21taskset08:25:143
1826799393,27cyclictest11428-21taskset08:25:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional