You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 06:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Fri May 01, 2026 00:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10205991213,108cyclictest0-21swapper/021:32:160
10205991203,77cyclictest22865-21awk19:37:330
9379211912,69sleep08725-21/usr/sbin/munin18:57:480
102059911729,74cyclictest0-21swapper/020:22:220
102059911729,74cyclictest0-21swapper/020:22:220
10205991133,21cyclictest17223-21latency_hist21:02:080
102059911241,63cyclictest0-21swapper/021:17:200
102059911230,55cyclictest5045-21ssh22:28:030
102059911230,55cyclictest5045-21ssh22:28:030
10205991112,24cyclictest26095-21rm23:58:330
102059911041,60cyclictest7563-21latency_hist23:27:090
10205991103,28cyclictest14892-21ssh23:38:050
10205991093,23cyclictest6107-21grep20:27:200
10205991092,26cyclictest14276-21cp19:12:100
10205991092,26cyclictest14276-21cp19:12:100
102059910838,62cyclictest0-21swapper/021:52:220
102059910838,62cyclictest0-21swapper/021:52:220
10205991083,27cyclictest2964-21ssh00:14:180
10205991083,27cyclictest2964-21ssh00:14:180
10205991082,25cyclictest9269-21rm00:25:120
10205991074,24cyclictest3409-21ssh22:26:520
102059910634,59cyclictest0-21swapper/021:22:280
10205991063,24cyclictest19577-21rm21:05:370
10205991063,24cyclictest19577-21rm21:05:370
10205991052,26cyclictest20691-21ssh22:54:060
10205991052,24cyclictest22244-21rm23:51:520
10205991052,24cyclictest22244-21rm23:51:520
10205991043,23cyclictest22633-21rm23:52:100
10205991043,23cyclictest22633-21rm23:52:100
10205991042,24cyclictest26301-21ssh23:03:030
10205991042,24cyclictest26301-21ssh23:03:030
10205991042,23cyclictest11647-21sh22:39:010
10205991042,23cyclictest11647-21sh22:39:010
10205991033,7cyclictest0-21swapper/000:07:170
10205991033,7cyclictest0-21swapper/000:07:170
10205991032,6cyclictest0-21swapper/000:17:210
10205991032,30cyclictest30277-21apt-get22:17:160
10205991032,25cyclictest9530-21runrttasks23:29:560
10205991032,22cyclictest22329-21ssh21:09:540
10205991023,7cyclictest0-21swapper/020:07:410
10205991023,21cyclictest23047-21ssh22:04:370
10205991022,6cyclictest0-21swapper/022:57:190
10205991022,22cyclictest28720-21sh22:13:390
10205991022,22cyclictest28720-21sh22:13:390
10205991013,24cyclictest13156-21rm23:36:370
10205991012,24cyclictest13744-21ssh21:48:480
10205991012,23cyclictest7949-21ssh21:39:240
10205991012,23cyclictest30680-21rm23:11:540
10205991012,23cyclictest25568-21/usr/sbin/munin19:47:410
10205991012,23cyclictest25568-21/usr/sbin/munin19:47:410
102149910047,46cyclictest17167-21latency23:42:332
102149910038,9cyclictest0-21swapper/222:27:072
102149910026,24cyclictest13226-21cut00:32:082
102059910071,20cyclictest26180-21gpgv21:17:090
10205991003,6cyclictest0-21swapper/019:27:150
10205991002,24cyclictest26682-21ssh22:11:240
10205991002,24cyclictest10987-21ssh21:44:290
10205991002,23cyclictest18486-21ssh23:44:450
10205991002,23cyclictest14148-21apt-get20:52:170
10205991002,23cyclictest14148-21apt-get20:52:170
1020599993,7cyclictest0-21swapper/019:57:170
1020599993,64cyclictest9512-21latency_hist22:37:080
1020599993,64cyclictest9512-21latency_hist22:37:080
1020599993,25cyclictest28764-21head00:02:400
1020599992,23cyclictest12619-21ssh00:31:100
1021499982,35cyclictest23263-21grep19:37:422
1020599983,23cyclictest3876-21taskset20:18:160
1020599983,23cyclictest3876-21taskset20:18:160
10205999737,52cyclictest0-21swapper/019:32:270
1020599972,6cyclictest0-21swapper/023:22:180
94362958,61sleep19491-21cat18:57:491
10214999547,40cyclictest0-21swapper/223:57:052
10214999547,40cyclictest0-21swapper/223:57:052
1020599952,8cyclictest0-21swapper/019:07:220
1020599952,27cyclictest12518-21taskset20:47:170
10214999441,9cyclictest0-21swapper/200:17:452
10214999434,12cyclictest0-21swapper/221:32:282
10214999418,23cyclictest12977-21cat21:47:392
1020599944,81cyclictest18277-21apt-get21:57:160
1020599943,7cyclictest0-21swapper/019:17:170
1020599942,6cyclictest0-21swapper/022:47:210
1020599942,6cyclictest0-21swapper/022:47:210
1020599942,25cyclictest32206-21/usr/sbin/munin23:12:440
1020599942,23cyclictest13309-21fschecks_count22:42:260
874129313,51sleep28744-21apt-get18:57:172
10214999369,18cyclictest13850-21apt-key19:12:072
10214999340,7cyclictest0-21swapper/222:12:332
10214999340,7cyclictest0-21swapper/222:12:332
10214999332,11cyclictest0-21swapper/220:57:432
1021499932,36cyclictest29006-21fschecks_count19:57:252
1020599933,6cyclictest0-21swapper/021:27:150
1020599933,6cyclictest0-21swapper/021:27:150
1020599933,66cyclictest9019-21modprobe20:37:050
1020599933,25cyclictest30911-21awk20:02:320
1020599932,29cyclictest27838-21/usr/sbin/munin19:52:380
1021499924,37cyclictest10537-21latency_hist23:32:082
10214999231,12cyclictest0-21swapper/222:17:412
10214999230,15cyclictest0-21swapper/220:37:212
10205999231,55cyclictest0-21swapper/019:02:280
10205999231,55cyclictest0-21swapper/019:02:280
10214999136,8cyclictest0-21swapper/222:10:522
1021499913,43cyclictest27245-21taskset21:18:242
10214999127,24cyclictest32468-21latency_hist21:27:082
1021499912,35cyclictest18154-21rm21:02:382
1021499912,35cyclictest18154-21rm21:02:382
1020599913,6cyclictest0-21swapper/020:12:200
1020599913,23cyclictest17791-21fschecks_time19:22:270
1020599912,22cyclictest2387-21/usr/sbin/munin23:17:380
10214999042,41cyclictest11288-21sh22:38:502
10214999042,41cyclictest11288-21sh22:38:502
10214999027,13cyclictest0-21swapper/222:02:312
1020599903,6cyclictest0-21swapper/019:42:200
10214998943,34cyclictest24747-21modprobe23:02:052
1021499893,39cyclictest1461-21fschecks_time20:12:262
1021499892,34cyclictest7395-21latency_hist20:32:082
10214998861,19cyclictest30505-21latency_hist00:07:072
1021499883,39cyclictest6305-21ssh21:37:212
1021499883,38cyclictest15882-21latency_hist19:17:092
1021499883,33cyclictest12248-21latency_hist20:47:082
10214998814,28cyclictest2223-21cat00:12:402
10214998814,28cyclictest2223-21cat00:12:402
10214998733,9cyclictest0-21swapper/221:42:382
1021499873,32cyclictest3744-21unixbench_multi20:17:412
1021499873,32cyclictest3744-21unixbench_multi20:17:412
1021499873,32cyclictest26526-21apt-config21:17:102
1021499872,34cyclictest14003-21latency_hist20:52:082
10214998719,24cyclictest22105-21modprobe19:37:052
10209998773,10cyclictest10196-21cyclictest00:29:191
10209998765,15cyclictest365-21ntpd22:57:441
1020599872,28cyclictest11470-21/usr/sbin/munin20:42:370
1021499864,33cyclictest25683-21cstates19:47:232
1021499864,33cyclictest25683-21cstates19:47:232
1021499864,33cyclictest20613-21latency_hist19:32:082
1021499863,33cyclictest7930-21fschecks_time20:32:262
1021499854,34cyclictest28046-21/usr/sbin/munin23:07:222
1021499853,38cyclictest25438-21latency_hist19:47:082
1021499853,37cyclictest15618-21basename21:52:292
1021499853,37cyclictest15618-21basename21:52:292
1021499852,36cyclictest16180-21fschecks_time19:17:302
10209998562,17cyclictest352-21runrttasks21:32:351
1020599852,7cyclictest0-21swapper/020:37:200
10214998463,13cyclictest30911-21ssh00:07:272
10214998463,13cyclictest30911-21ssh00:07:272
10209998469,11cyclictest10196-21cyclictest20:02:051
10214998360,15cyclictest26443-21kworker/u18:2-rpciod23:17:412
1021499835,33cyclictest18361-21cpuspeed21:57:212
1021499834,32cyclictest352-21runrttasks23:06:132
1021499834,32cyclictest352-21runrttasks23:06:132
10209998352,8cyclictest181rcu_preempt20:57:221
10209998346,14cyclictest31-21ksoftirqd/121:38:251
1021499824,40cyclictest28753-21latency_hist19:57:082
1021499824,36cyclictest17111-21diskmemload22:27:392
1021499824,36cyclictest17111-21diskmemload22:27:392
1021499813,36cyclictest7744-21timerandwakeup22:32:412
1021499813,36cyclictest7744-21timerandwakeup22:32:412
10214998131,9cyclictest0-21swapper/221:30:462
10214998131,9cyclictest0-21swapper/221:30:462
10214998131,10cyclictest0-21swapper/219:02:242
10214998131,10cyclictest0-21swapper/219:02:242
10209998158,16cyclictest3112-21latency_hist21:32:091
10209998158,16cyclictest3112-21latency_hist21:32:091
10209998060,14cyclictest1740-21sed20:12:331
10209998058,16cyclictest352-21runrttasks19:27:261
10209998049,8cyclictest181rcu_preempt20:37:201
1021499793,34cyclictest352-21runrttasks22:46:172
10209997955,18cyclictest17974-21sh22:50:011
10209997955,18cyclictest17974-21sh22:50:011
1020999792,54cyclictest4649-21grep20:22:281
1020999792,54cyclictest4649-21grep20:22:281
10209997859,14cyclictest10196-21cyclictest21:04:491
10209997859,14cyclictest10196-21cyclictest21:04:491
10209997859,12cyclictest352-21runrttasks19:22:401
1021499774,30cyclictest15648-21modprobe19:17:052
1021499764,35cyclictest15480-21cron20:57:042
1021499764,35cyclictest15480-21cron20:57:042
10209997656,14cyclictest4322-21latency_hist23:22:081
10209997656,13cyclictest10196-21cyclictest23:52:361
10209997656,13cyclictest10196-21cyclictest23:52:361
1020999755,61cyclictest26703-21apt-get21:17:501
10209997552,13cyclictest0-21swapper/121:17:061
10209997551,9cyclictest181rcu_preempt00:12:381
10209997551,9cyclictest181rcu_preempt00:12:381
10209997451,14cyclictest0-21swapper/122:17:271
10209997448,20cyclictest8593-21kworker/u18:1+nfsiod00:25:081
10209997448,18cyclictest9645-21apt-key22:37:071
10209997448,18cyclictest9645-21apt-key22:37:071
1020999743,62cyclictest13788-21apt-get19:12:251
1020999743,62cyclictest13788-21apt-get19:12:251
1020999743,40cyclictest10853-21df23:32:261
10214997356,11cyclictest1-21systemd23:27:062
10214997356,11cyclictest1-21systemd23:27:062
10214997352,15cyclictest19928-21fschecks_time23:47:272
10214997352,15cyclictest19928-21fschecks_time23:47:272
10214997351,15cyclictest17111-21diskmemload22:56:072
10209997359,9cyclictest31-21ksoftirqd/122:30:281
10209997359,9cyclictest31-21ksoftirqd/122:30:281
10209997356,11cyclictest17111-21diskmemload22:12:081
10209997345,16cyclictest0-21swapper/123:22:421
10209997345,16cyclictest0-21swapper/123:22:421
10209997327,35cyclictest6689-21kworker/u18:0+nfsiod23:38:561
10209997257,10cyclictest32252-21kworker/u17:0+xprtiod23:15:281
10209997254,12cyclictest26941-21sendmail19:52:061
10209997254,12cyclictest26941-21sendmail19:52:061
10209997249,15cyclictest2374-21systemd-journal21:56:461
10209997249,15cyclictest2374-21systemd-journal21:56:461
10209997240,23cyclictest11726-21cron19:07:061
10209997240,23cyclictest11726-21cron19:07:061
10214997149,14cyclictest17929-21sh22:49:522
10214997149,14cyclictest17929-21sh22:49:522
10209997154,11cyclictest26624-21rm00:00:011
10209997152,12cyclictest15050-21latency_hist21:52:071
10209997148,16cyclictest11005-21sh22:38:431
10209997148,16cyclictest11005-21sh22:38:431
10209997146,10cyclictest181rcu_preempt21:10:341
1020999713,57cyclictest17264-21idleruntime-cro19:22:051
10214997057,9cyclictest24804-21kworker/u17:1-rpciod23:12:562
1020999707,23cyclictest8315-21mailstats20:32:371
1020999706,41cyclictest31473-21/usr/sbin/munin00:07:391
1020999706,41cyclictest31473-21/usr/sbin/munin00:07:391
10209997053,9cyclictest0-21swapper/122:02:311
10209997034,28cyclictest13775-21kworker/u18:1+nfsiod22:25:031
10214996946,15cyclictest0-21swapper/223:37:342
1020999695,21cyclictest31-21ksoftirqd/123:47:191
1020999695,21cyclictest31-21ksoftirqd/123:47:191
10209996951,10cyclictest4573-21ssh00:17:301
10209996949,10cyclictest0-21swapper/121:42:281
10209996910,37cyclictest24879-21grep19:42:411
1972826812,8sleep10-21swapper/121:58:431
10214996846,16cyclictest5788-21sh20:27:052
10214996846,16cyclictest5788-21sh20:27:052
10209996849,10cyclictest0-21swapper/119:52:421
10209996845,8cyclictest181rcu_preempt23:47:041
1020999683,45cyclictest13994-21latency_hist19:12:081
10209996750,10cyclictest13789-21diskmemload22:42:381
1020999674,42cyclictest352-21runrttasks20:44:461
1022399663,44cyclictest27129-21chrt21:18:203
10214996652,8cyclictest0-21swapper/223:32:352
10214996645,14cyclictest951-21munin-plugin-st20:12:082
1020999664,34cyclictest352-21runrttasks22:53:581
1020999664,34cyclictest27323-21cpu22:12:211
1020999664,34cyclictest27323-21cpu22:12:211
10214996549,10cyclictest2374-21systemd-journal19:23:472
10209996551,10cyclictest10196-21cyclictest20:27:281
1020999654,40cyclictest22623-21fschecks_time19:37:271
10209996543,12cyclictest0-21swapper/123:02:281
10209996543,12cyclictest0-21swapper/123:02:281
10209996540,10cyclictest181rcu_preempt21:25:401
10209996539,19cyclictest298-21in:imuxsock00:02:461
10209996534,9cyclictest181rcu_preempt20:12:081
1020999652,42cyclictest4175-21modprobe20:22:051
1020999652,42cyclictest4175-21modprobe20:22:051
1020999652,28cyclictest15502-21munin-run20:57:051
1020999652,28cyclictest15502-21munin-run20:57:051
10214996450,8cyclictest0-21swapper/221:07:242
10209996443,15cyclictest1-21systemd19:32:461
1020999643,41cyclictest10480-21latency_hist23:32:071
1020999643,32cyclictest31408-21unixbench_singl20:02:421
1022399634,46cyclictest16375-21taskset21:52:443
1022399634,46cyclictest16375-21taskset21:52:443
10214996350,9cyclictest10196-21cyclictest00:26:242
1020999634,45cyclictest28196-21cstates23:07:231
1020999633,44cyclictest14106-21latency_hist20:52:091
1022399623,40cyclictest917-21kworker/3:1+events22:11:173
10214996246,10cyclictest277-21dbus-daemon00:02:062
1022399612,42cyclictest19024-21taskset22:52:093
1022399605,39cyclictest23208-21chrt22:05:043
1022399602,38cyclictest917-21kworker/3:1+events23:42:193
1022399593,41cyclictest11523-21chrt23:32:423
1022399593,38cyclictest917-21kworker/3:1+events20:22:393
1022399593,38cyclictest917-21kworker/3:1+events20:22:393
1022399584,37cyclictest7853-21taskset23:27:273
1022399583,39cyclictest1196-21chrt00:12:093
10223995828,20cyclictest10003-21chrt21:42:393
1022399582,38cyclictest917-21kworker/3:1+events20:47:093
1022399582,38cyclictest917-21kworker/3:1+events00:02:253
1022399573,36cyclictest26564-21chrt21:17:103
10214995744,7cyclictest0-21swapper/220:02:262
1022399563,39cyclictest9239-21chrt00:25:063
1022399563,37cyclictest21922-21taskset21:08:473
1022399563,36cyclictest4108-21taskset00:17:073
1022399563,36cyclictest4108-21taskset00:17:073
1022399563,34cyclictest3713-21chrt22:27:083
1022399562,38cyclictest4341-21taskset21:32:423
1022399554,34cyclictest21138-21chrt22:02:083
1022399552,35cyclictest7431-21chrt21:37:573
98372540,6sleep30-21swapper/320:37:363
1022399546,35cyclictest18807-21chrt21:03:283
1022399546,35cyclictest18807-21chrt21:03:283
1022399543,39cyclictest928-21chrt00:12:053
1022399543,35cyclictest9998-21chrt22:37:093
1022399543,35cyclictest9998-21chrt22:37:093
1022399542,37cyclictest29083-21chrt19:57:263
1022399533,38cyclictest22946-21chrt23:52:283
1022399533,38cyclictest22946-21chrt23:52:283
1022399533,37cyclictest31128-21chrt23:12:203
1022399533,37cyclictest12788-21chrt19:07:333
1022399533,34cyclictest11406-21chrt00:27:453
1022399532,38cyclictest11356-21taskset19:02:433
1022399532,38cyclictest11356-21taskset19:02:433
10223995314,18cyclictest0-21swapper/323:52:073
10223995314,18cyclictest0-21swapper/323:52:073
1022399523,36cyclictest29191-21taskset23:07:463
1022399522,39cyclictest848-21taskset21:27:363
1022399522,39cyclictest848-21taskset21:27:363
1022399514,33cyclictest23292-21taskset19:37:423
1022399513,34cyclictest30883-21chrt21:22:583
1022399513,34cyclictest1741-21chrt20:12:333
1022399513,30cyclictest27383-21chrt22:12:213
1022399513,30cyclictest27383-21chrt22:12:213
1022399512,36cyclictest14756-21chrt20:52:393
1022399512,36cyclictest14756-21chrt20:52:393
1022399512,33cyclictest32163-21chrt20:07:233
10223995115,27cyclictest0-21swapper/319:12:223
10223995115,27cyclictest0-21swapper/319:12:223
1022399506,32cyclictest2420-21chrt23:17:393
1022399504,36cyclictest12627-21chrt21:47:293
1022399503,35cyclictest4602-21chrt23:22:123
1022399503,35cyclictest4602-21chrt23:22:123
1022399503,34cyclictest22325-21taskset19:37:093
1022399503,34cyclictest15593-21chrt22:46:423
1022399503,31cyclictest9013-21chrt22:35:433
1022399503,31cyclictest9013-21chrt22:35:433
1022399502,34cyclictest7433-21chrt20:32:053
1022399502,34cyclictest22694-21taskset22:57:373
1022399502,32cyclictest25965-21chrt23:58:113
1022399502,32cyclictest16010-21chrt22:47:113
1022399502,32cyclictest16010-21chrt22:47:113
1022399493,34cyclictest15908-21chrt20:57:243
1022399493,32cyclictest4191-21chrt20:22:053
1022399493,32cyclictest4191-21chrt20:22:053
1022399492,35cyclictest32485-21taskset22:20:483
1022399492,29cyclictest917-21kworker/3:1+events23:37:293
1022399483,32cyclictest25555-21chrt19:47:093
1022399472,35cyclictest25306-21taskset23:02:273
1022399472,35cyclictest25306-21taskset23:02:273
1022399472,32cyclictest4313-21taskset00:17:203
905224610,22sleep30-21swapper/318:57:343
1022399463,32cyclictest4205-21taskset22:27:293
1022399463,32cyclictest4205-21taskset22:27:293
1022399452,30cyclictest19524-21chrt19:27:293
1022399443,29cyclictest28220-21taskset19:52:533
1022399433,30cyclictest17299-21chrt19:22:073
1022399423,28cyclictest18603-21taskset19:23:493
1022399413,28cyclictest26693-21taskset19:49:163
1022399413,28cyclictest26693-21taskset19:49:163
1022399413,27cyclictest30594-21chrt20:02:243
1022399413,25cyclictest7905-21taskset20:32:263
1022399412,29cyclictest13050-21chrt20:47:363
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional