You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 15:11
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Sun Jul 26, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12460991232,42cyclictest16572-21ssh11:31:260
124609912112,25cyclictest26441-21rm10:53:170
12460991194,26cyclictest16817-21/usr/sbin/munin10:37:480
12460991183,23cyclictest6542-21sh09:26:530
12314211612,68sleep112318-21cron06:56:131
12314211612,68sleep112318-21cron06:56:131
12460991123,14cyclictest0-21swapper/006:56:170
12460991123,14cyclictest0-21swapper/006:56:170
1219821106,15sleep20-21swapper/206:54:432
1219821106,15sleep20-21swapper/206:54:432
124759910850,9cyclictest0-21swapper/211:51:322
12460991073,25cyclictest22693-21date09:01:160
12460991064,96cyclictest0-21swapper/007:01:270
12460991053,75cyclictest13578-21apt-get11:26:300
12460991043,30cyclictest15078-21http10:36:140
12460991043,26cyclictest11238-21ssh12:16:220
124609910415,23cyclictest22981-21sh10:47:350
12460991033,27cyclictest3562-21ssh11:09:560
12460991033,27cyclictest3562-21ssh11:09:560
12460991033,26cyclictest8767-21sh11:17:460
12460991032,25cyclictest352-21runrttasks07:20:040
1620721020,5sleep30-21swapper/309:43:103
12460991022,31cyclictest11389-21latency_hist08:31:160
12460991022,26cyclictest25729-21/usr/sbin/munin11:46:330
12460991022,26cyclictest25729-21/usr/sbin/munin11:46:330
12482991013,60cyclictest0-21swapper/312:07:263
12460991012,23cyclictest2550-21/usr/sbin/munin09:21:310
12460991012,23cyclictest2550-21/usr/sbin/munin09:21:310
12460991012,23cyclictest11628-21fschecks_count09:36:350
12460991012,23cyclictest11628-21fschecks_count09:36:350
1296321000,7sleep10-21swapper/106:56:371
1296321000,7sleep10-21swapper/106:56:371
124669910078,14cyclictest5866-21diskmemload12:07:321
124609910011,25cyclictest10833-21sh12:15:550
1246099992,31cyclictest2180-21ls08:02:160
1246099983,28cyclictest26757-21latency_hist10:01:170
1246099973,24cyclictest21752-21vmstat09:51:520
1246099972,86cyclictest11127-21sshd09:36:140
1246099972,27cyclictest1728-21latency_hist12:01:160
1246099972,27cyclictest1728-21latency_hist12:01:160
1246099972,25cyclictest5179-21/usr/sbin/munin11:11:490
1246099972,24cyclictest352-21runrttasks11:43:300
1246099972,24cyclictest17038-21sh09:45:310
1246099963,31cyclictest16309-21/usr/sbin/munin07:06:490
1246099963,27cyclictest9369-21/usr/sbin/munin10:26:310
12460999631,60cyclictest12451-21cyclictest11:25:450
12460999631,60cyclictest12451-21cyclictest11:25:450
1246099962,26cyclictest5121-21chrt10:18:380
1246099962,23cyclictest15389-21cat08:41:500
1246099962,12cyclictest32226-21fschecks_time09:16:390
12460999611,22cyclictest19777-21sh10:42:040
12460999611,22cyclictest19777-21sh10:42:040
1246699953,64cyclictest13603-21ssh11:26:301
1246099957,10cyclictest0-21swapper/008:11:170
1246099953,36cyclictest28573-21taskset11:51:290
1246099953,34cyclictest4747-21awk12:06:130
1246099953,31cyclictest30003-21unixbench_singl09:11:520
1246099953,31cyclictest30003-21unixbench_singl09:11:520
1246099953,24cyclictest6674-21/usr/sbin/munin08:16:390
1246099952,23cyclictest27246-21runrttasks07:40:230
12460999521,65cyclictest21618-21/usr/sbin/munin07:21:410
12460999521,65cyclictest21618-21/usr/sbin/munin07:21:410
12466999447,41cyclictest15992-21unixbench_multi09:42:461
1246099943,31cyclictest24630-21fschecks_time07:31:340
1246099943,18cyclictest27597-21apt-get07:41:270
1246099942,31cyclictest18068-21ls08:51:150
1246099933,10cyclictest22914-21ssh09:01:270
1246099932,31cyclictest5309-21ssh12:06:250
1246099932,31cyclictest30018-21timerandwakeup07:46:490
1246099932,21cyclictest28873-21ssh10:04:250
12460999314,25cyclictest16788-21rm12:25:520
1246699922,30cyclictest6619-21taskset09:27:051
1246699922,24cyclictest4487-21taskset08:07:061
1246099923,9cyclictest0-21swapper/011:01:520
12460999235,51cyclictest0-21swapper/007:31:160
1246099923,29cyclictest7422-21/usr/sbin/munin10:21:510
1246099923,29cyclictest7422-21/usr/sbin/munin10:21:510
12460999226,58cyclictest0-21swapper/008:36:150
12460999213,22cyclictest18183-21tune2fs07:11:370
1246099914,29cyclictest32678-21latency_hist10:11:160
1246099914,29cyclictest28363-21/usr/sbin/munin10:56:430
1246099914,22cyclictest22311-21grep11:41:150
1246099913,78cyclictest1684-21ssh10:12:210
1246099913,28cyclictest31089-21fschecks_time07:51:370
1246099913,28cyclictest31089-21fschecks_time07:51:370
1246099913,25cyclictest28807-21grep09:11:170
1246099913,20cyclictest6625-21latency_hist08:16:170
1246099912,29cyclictest1744-21latency_hist08:01:170
1246099903,25cyclictest17907-21fschecks_count09:46:370
1246099902,26cyclictest13964-21timerandwakeup08:36:490
12466998946,35cyclictest0-21swapper/110:06:511
1246699893,61cyclictest1326-21apt-get08:01:271
1246099892,28cyclictest18943-21grep08:51:460
1246099892,28cyclictest18943-21grep08:51:460
12466998843,37cyclictest0-21swapper/107:41:131
12460998822,58cyclictest0-21swapper/006:56:300
12460998822,58cyclictest0-21swapper/006:56:300
1248299874,41cyclictest0-21swapper/309:56:363
12466998745,35cyclictest0-21swapper/111:46:471
12466998745,35cyclictest0-21swapper/111:46:471
1246699872,29cyclictest21976-21cat10:46:381
12466998660,18cyclictest19605-21diskmemload09:51:171
12466998659,18cyclictest11085-21ntpd09:32:091
12466998647,33cyclictest0-21swapper/111:01:521
12466998645,33cyclictest0-21swapper/109:16:531
12466998642,36cyclictest0-21swapper/109:06:131
1246699862,38cyclictest26322-21/usr/sbin/munin09:06:371
12475998561,16cyclictest14505-21cut12:21:362
12466998547,31cyclictest20402-21switchtime11:36:501
12466998544,34cyclictest17095-21sh11:31:431
1246699853,36cyclictest6582-21/usr/sbin/munin10:21:351
1246699853,36cyclictest6582-21/usr/sbin/munin10:21:351
1246699852,76cyclictest29876-21cut07:46:471
1246099852,25cyclictest8505-21fschecks_count08:21:360
1246099852,25cyclictest8505-21fschecks_count08:21:360
12466998461,15cyclictest2374-21systemd-journal11:09:361
12466998461,15cyclictest2374-21systemd-journal11:09:361
1246699843,39cyclictest2425-21/usr/sbin/munin09:21:241
1246699843,39cyclictest2425-21/usr/sbin/munin09:21:241
1246699843,36cyclictest8295-21apt-get08:21:251
1246699843,36cyclictest8295-21apt-get08:21:251
1246699842,40cyclictest15052-21latency_hist10:36:181
1246699842,35cyclictest21539-21/usr/sbin/munin09:51:491
12466998344,32cyclictest0-21swapper/107:16:371
12466998339,39cyclictest0-21swapper/109:41:151
12466998339,39cyclictest0-21swapper/109:41:151
1246699833,6cyclictest0-21swapper/111:51:261
12466998334,34cyclictest30244-21kworker/u17:3+events_unbound07:56:171
1246699832,7cyclictest0-21swapper/107:41:281
12466998243,32cyclictest0-21swapper/109:11:511
12466998243,32cyclictest0-21swapper/109:11:511
1246699823,6cyclictest0-21swapper/107:11:271
1246699823,40cyclictest13087-21/usr/sbin/munin10:31:451
1246699823,31cyclictest5543-21netstat08:11:421
1246699823,30cyclictest15179-21/usr/sbin/munin07:01:441
12466998231,44cyclictest27149-21sh10:01:361
12466998143,31cyclictest6747-21cpu08:16:321
1246699813,37cyclictest22995-21/usr/sbin/munin07:26:361
1246699813,34cyclictest25200-21latency_hist11:46:171
1246699813,32cyclictest32058-21sshd11:56:401
1246699813,32cyclictest32058-21sshd11:56:401
1246699812,38cyclictest824-21/usr/sbin/munin10:11:431
1246699812,36cyclictest18679-21grep10:41:321
1246699812,36cyclictest18679-21grep10:41:321
1246699812,35cyclictest14464-21ssh12:21:351
1246699812,34cyclictest15325-21/usr/sbin/munin08:41:491
12475998063,10cyclictest8947-21runrttasks08:21:472
12475998063,10cyclictest8947-21runrttasks08:21:472
12475998059,15cyclictest22696-21unixbench_singl10:46:522
12475998055,18cyclictest15242-21fschecks_time09:42:312
12466998055,10cyclictest181rcu_preempt12:16:161
1246699804,35cyclictest25266-21/usr/sbin/munin07:31:481
12466998038,35cyclictest0-21swapper/108:56:401
1246699803,7cyclictest0-21swapper/109:56:431
1246699803,35cyclictest13405-21/usr/sbin/munin08:36:361
1246699803,33cyclictest30491-21ls11:01:131
1246699803,30cyclictest9956-21/usr/sbin/munin10:26:441
1246699802,37cyclictest4720-21sh12:06:121
1246699802,34cyclictest17066-21/usr/sbin/munin08:47:381
12475997963,10cyclictest12451-21cyclictest11:16:472
1246699793,71cyclictest0-21swapper/108:31:231
1246699793,36cyclictest10751-21/usr/sbin/munin11:21:401
1246699793,36cyclictest10751-21/usr/sbin/munin11:21:401
1246699793,33cyclictest4686-21fschecks_time11:11:371
1246699792,33cyclictest13227-21taskset12:19:151
12475997860,11cyclictest23913-21kworker/u17:2-rpciod10:03:132
12466997851,18cyclictest6179-21latency_hist10:21:161
1246699782,32cyclictest31582-21switchtime07:51:491
1246699782,32cyclictest31582-21switchtime07:51:491
12475997759,10cyclictest19591-21munin-plugin-st08:56:132
12475997759,10cyclictest19591-21munin-plugin-st08:56:132
1246699773,7cyclictest0-21swapper/107:21:291
1246699773,7cyclictest0-21swapper/107:21:291
1246699773,36cyclictest18429-21taskset08:51:331
1246699773,36cyclictest18429-21taskset08:51:331
1246699773,33cyclictest10464-21ntp_states08:26:451
1246699772,36cyclictest9919-21runrttasks11:21:011
1246699772,34cyclictest24815-21acpi10:51:231
12475997656,14cyclictest13010-21latency_hist11:26:182
12475997656,14cyclictest13010-21latency_hist11:26:182
12475997655,15cyclictest25750-21sh11:46:362
12475997655,15cyclictest25750-21sh11:46:362
1246699762,35cyclictest16480-21df_inode07:06:341
12482997533,26cyclictest1472-21taskset08:01:153
12475997555,15cyclictest3779-21sh11:10:332
12475997555,15cyclictest3779-21sh11:10:332
12475997456,12cyclictest2374-21systemd-journal10:14:362
12475997452,15cyclictest16468-21latency_hist08:46:172
12475997451,16cyclictest16371-21latency_hist11:31:162
12475997439,13cyclictest12451-21cyclictest09:04:462
12475997358,9cyclictest39-21ksoftirqd/208:31:432
12475997356,11cyclictest2374-21systemd-journal12:13:352
12475997353,14cyclictest7275-21processes08:16:482
12475997352,14cyclictest4649-21cut11:11:372
12475997351,16cyclictest23775-21latency_hist09:56:182
12475997259,9cyclictest9911-21kworker/u17:2-xprtiod10:27:202
12475997254,10cyclictest0-21swapper/207:56:472
12475997253,13cyclictest32450-21sh11:03:222
12475997252,9cyclictest0-21swapper/209:36:132
12475997252,14cyclictest30569-21latency_hist11:01:152
12475997240,23cyclictest20146-21grep11:36:442
12482997130,28cyclictest32099-21taskset09:16:353
12475997156,9cyclictest39-21ksoftirqd/209:10:342
1247599706,14cyclictest16557-21apt-get11:31:282
12475997054,11cyclictest58-21kcompactd012:10:382
1248299693,45cyclictest0-21swapper/310:36:303
12482996925,29cyclictest15357-21chrt07:01:493
12475996954,8cyclictest0-21swapper/209:21:532
12475996954,8cyclictest0-21swapper/209:21:532
12475996947,16cyclictest32033-21cstates09:16:332
12475996926,11cyclictest12372-21kworker/u18:2-events_unbound08:16:172
1247599692,16cyclictest32038-21/usr/sbin/munin11:56:392
1247599692,16cyclictest32038-21/usr/sbin/munin11:56:392
12475996854,10cyclictest28587-21kworker/u17:3-writeback09:12:522
12475996854,10cyclictest28587-21kworker/u17:3-writeback09:12:522
12475996852,9cyclictest0-21swapper/209:46:512
12475996851,11cyclictest29923-21switchtime07:46:482
12475996849,13cyclictest2374-21systemd-journal10:54:352
1247599684,49cyclictest27368-21latency_hist07:41:152
1247599683,49cyclictest20355-21timerwakeupswit07:16:472
12475996749,12cyclictest4843-21systemd-run08:11:142
12475996747,14cyclictest0-21swapper/210:41:552
12475996747,14cyclictest0-21swapper/210:41:552
12475996739,12cyclictest28587-21kworker/u17:3-ext4-rsv-conversion09:31:162
1247599673,47cyclictest25460-21latency_hist11:46:172
1247599672,55cyclictest16886-21/usr/sbin/munin08:47:342
12482996627,27cyclictest5472-21taskset10:19:383
12475996643,16cyclictest2124-21fschecks_time08:02:152
1247599663,47cyclictest9928-21apt-get08:26:252
1247599662,54cyclictest23305-21apt-get09:56:302
12482996532,21cyclictest16675-21chrt07:06:393
12482996527,23cyclictest12748-21taskset08:34:413
12475996548,9cyclictest0-21swapper/208:36:492
1247599654,46cyclictest30321-21grep10:06:392
1247599654,41cyclictest11391-21/usr/sbin/munin09:36:322
1247599654,41cyclictest11391-21/usr/sbin/munin09:36:322
12475996535,8cyclictest181rcu_preempt07:56:132
12475996535,8cyclictest181rcu_preempt07:56:132
12482996430,22cyclictest12911-21chrt11:26:013
12482996430,22cyclictest12911-21chrt11:26:013
12482996429,22cyclictest21374-21taskset08:57:503
12482996428,23cyclictest15842-21chrt12:23:123
12482996427,25cyclictest22906-21taskset10:47:223
12482996427,25cyclictest17848-21taskset11:32:223
12482996426,25cyclictest31109-21taskset10:07:143
12482996425,25cyclictest2141-21taskset12:01:323
12482996424,24cyclictest22850-21taskset09:01:193
1248299642,33cyclictest0-21swapper/310:51:463
1247599646,42cyclictest352-21runrttasks07:43:412
12475996447,11cyclictest17048-21users10:37:502
1247599643,45cyclictest13555-21grep06:56:502
1247599643,45cyclictest13555-21grep06:56:502
12475996418,37cyclictest20231-21diskmemload08:56:402
12482996331,20cyclictest28619-21chrt10:03:413
12482996330,22cyclictest27821-21taskset09:09:053
12482996330,21cyclictest9808-21chrt11:20:423
12482996330,21cyclictest13350-21chrt12:19:353
12482996326,25cyclictest8812-21taskset12:11:413
12482996325,27cyclictest27975-21taskset11:50:333
12482996325,27cyclictest27975-21taskset11:50:333
12475996350,8cyclictest0-21swapper/210:31:222
1247599634,42cyclictest16960-21mailstats07:06:472
1247599632,44cyclictest14936-21fschecks_time07:01:382
12482996230,21cyclictest21810-21chrt11:40:083
12482996229,22cyclictest6552-21taskset10:21:323
12482996229,22cyclictest6552-21taskset10:21:323
12482996229,22cyclictest24141-21chrt11:43:163
12482996229,21cyclictest16055-21chrt08:45:553
12482996228,20cyclictest2453-21chrt09:21:293
12482996228,20cyclictest2453-21chrt09:21:293
12482996227,22cyclictest6714-21chrt09:27:213
12482996225,25cyclictest6222-21taskset11:14:013
12482996225,22cyclictest19031-21chrt07:13:203
1247599629,32cyclictest0-21swapper/207:11:372
12475996246,9cyclictest0-21swapper/207:21:352
12475996246,9cyclictest0-21swapper/207:21:352
12475996246,11cyclictest3709-21kworker/u18:1-rpciod12:03:322
12475996244,9cyclictest181rcu_preempt10:16:252
1247599623,43cyclictest11656-21interrupts12:16:362
1247599623,34cyclictest24628-21fschecks_time07:31:342
1248299613,46cyclictest10323-21taskset09:34:063
12482996129,20cyclictest19892-21chrt10:42:243
12482996129,20cyclictest19892-21chrt10:42:243
12482996128,22cyclictest15767-21chrt11:29:543
12482996128,21cyclictest12984-21chrt10:31:423
12482996128,20cyclictest28975-21chrt10:56:593
12482996128,20cyclictest25126-21chrt07:31:473
12482996127,22cyclictest7891-21chrt08:19:383
12482996127,22cyclictest18226-21taskset08:51:223
12482996127,22cyclictest18226-21taskset08:51:223
12482996127,21cyclictest24089-21chrt07:31:063
12482996123,25cyclictest28624-21taskset07:42:243
1247599613,44cyclictest6438-21cpu10:21:302
1247599613,44cyclictest6438-21cpu10:21:302
12482996031,19cyclictest32101-21chrt11:56:413
12482996031,19cyclictest32101-21chrt11:56:413
12482996029,20cyclictest1967-21chrt10:13:103
12482996028,20cyclictest16634-21chrt08:46:283
12482996027,21cyclictest31469-21chrt07:51:483
12482996027,21cyclictest31469-21chrt07:51:483
12482996027,21cyclictest12362-21chrt09:36:493
12482996027,21cyclictest12362-21chrt09:36:493
12482996027,20cyclictest12829-21taskset06:56:353
12482996027,20cyclictest12829-21taskset06:56:353
12482996027,20cyclictest10123-21chrt08:26:353
12482996026,22cyclictest27073-21chrt07:38:063
12482996026,21cyclictest5090-21chrt08:11:283
12482996025,23cyclictest29520-21taskset07:46:373
12482996021,25cyclictest20626-21taskset07:18:103
1247599605,41cyclictest352-21runrttasks07:28:432
12482995929,19cyclictest21158-21chrt07:21:303
12482995929,19cyclictest21158-21chrt07:21:303
12482995927,21cyclictest29720-21chrt11:52:033
12482995927,21cyclictest2892-21taskset11:08:023
12482995927,21cyclictest2892-21taskset11:08:023
12482995927,20cyclictest9297-21chrt08:22:013
12482995927,20cyclictest9297-21chrt08:22:013
12482995926,21cyclictest4627-21chrt08:08:573
12482995925,22cyclictest11771-21chrt10:30:283
12482995828,19cyclictest20211-21chrt09:50:563
12482995827,20cyclictest31611-21chrt11:01:503
12482995819,26cyclictest29049-21taskset09:11:323
12482995819,26cyclictest29049-21taskset09:11:323
12482995727,20cyclictest20650-21chrt09:51:253
12482995725,20cyclictest2889-21chrt08:02:413
12482995724,21cyclictest14229-21chrt08:38:013
1236525412,25sleep30-21swapper/306:56:153
1236525412,25sleep30-21swapper/306:56:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional