You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 11:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Mon Jun 08, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351780,0ktimersoftd/39920-21perf18:53:103
1643323624,0sleep10-21swapper/118:52:491
1625923424,0sleep20-21swapper/218:50:462
1611223127,0sleep00-21swapper/018:49:040
165549976,0cyclictest29753-21ssh23:42:330
35160,0ktimersoftd/30-21swapper/322:56:363
35160,0ktimersoftd/30-21swapper/322:49:213
339960,0migration/36830-21sh21:27:463
339960,0migration/330319-21sh23:02:563
339960,0migration/328603-21sh00:20:183
339960,0migration/321580-21sh22:15:123
339960,0migration/321510-21sh21:54:273
29725060,0irq/27-eth0-tx-19768-21munin-node20:08:320
29715060,0irq/26-eth0-rx-4178-21python23:03:413
259960,0migration/222320-21sh23:38:322
165579960,0cyclictest0-21swapper/221:50:002
165549960,0cyclictest0-21swapper/021:00:000
149960,0migration/020936-21sh23:35:210
149960,0migration/020497-21sh23:54:200
149960,0migration/012995-21sh22:49:140
339950,0migration/39572-21irqrtprio20:48:353
339950,0migration/37261-21kthreadcore19:33:363
339950,0migration/332311-21if_eth021:43:343
339950,0migration/327671-21munin-node23:58:393
339950,0migration/325007-21munin-node19:23:333
339950,0migration/32317-21kthreadcore20:18:373
339950,0migration/320896-21sh21:13:393
339950,0migration/319999-21cpuspeed_turbos20:08:333
339950,0migration/317569-21taskset19:18:013
339950,0migration/316339-21kthreadcore20:28:363
339950,0migration/315643-21munin-node23:13:343
339950,0migration/315170-21aten_r1power_po20:53:303
339950,0migration/313700-21taskset21:09:553
339950,0migration/313647-21latency_hist00:13:193
339950,0migration/313115-21runrttasks23:10:193
339950,0migration/310072-21kthreadcore20:23:373
29725050,0irq/27-eth0-tx-6439-21munin-node23:08:412
29725050,0irq/27-eth0-tx-5831-21munin-node00:08:402
29725050,0irq/27-eth0-tx-31638-21cpuspeed_turbos21:43:312
29725050,0irq/27-eth0-tx-31200-21irqcore00:03:352
29725050,0irq/27-eth0-tx-25510-21munin-node19:48:400
29725050,0irq/27-eth0-tx-15066-21munin-node20:53:340
29725050,0irq/27-eth0-tx-14398-21ssh22:12:352
29725050,0irq/27-eth0-tx-13068-21ssh23:30:482
29725050,0irq/27-eth0-tx-0-21swapper/223:19:252
29725050,0irq/27-eth0-tx-0-21swapper/222:52:462
29725050,0irq/27-eth0-tx-0-21swapper/221:58:352
29725050,0irq/27-eth0-tx-0-21swapper/221:53:372
29725050,0irq/27-eth0-tx-0-21swapper/019:43:520
29725050,0irq/27-eth0-tx-0-21swapper/019:33:310
29725050,0irq/27-eth0-tx-0-21swapper/019:08:450
29725050,0irq/27-eth0-tx-0-21swapper/019:08:450
29715050,0irq/26-eth0-rx-9503-21kthreadcore21:28:363
29715050,0irq/26-eth0-rx-7151-21grep22:08:323
29715050,0irq/26-eth0-rx-6453-21diskstats23:48:323
29715050,0irq/26-eth0-rx-5950-21latency_hist23:28:193
29715050,0irq/26-eth0-rx-4368-21open_inodes22:03:393
29715050,0irq/26-eth0-rx-30105-21grep20:13:373
29715050,0irq/26-eth0-rx-29084-21sed22:18:433
29715050,0irq/26-eth0-rx-23565-21grep22:38:343
29715050,0irq/26-eth0-rx-13226-21ssh23:31:153
29715050,0irq/26-eth0-rx-12498-21mdstat21:48:383
29715050,0irq/26-eth0-rx-0-21swapper/323:19:273
29715050,0irq/26-eth0-rx-0-21swapper/322:45:133
29715050,0irq/26-eth0-rx-0-21swapper/322:28:593
29715050,0irq/26-eth0-rx-0-21swapper/321:38:363
29715050,0irq/26-eth0-rx-0-21swapper/321:35:503
29715050,0irq/26-eth0-rx-0-21swapper/320:59:343
29715050,0irq/26-eth0-rx-0-21swapper/319:48:323
259950,0migration/25193-21runrttasks22:24:502
259950,0migration/227228-21kthreadcore19:48:362
259950,0migration/226761-21munin-node20:33:492
259950,0migration/219864-21missed_timers23:13:372
259950,0migration/218669-21kthreadcore23:53:372
259950,0migration/215490-21hwlatdetect22:53:342
249950,0watchdog/212742-21needreboot21:28:382
179950,0migration/130390-21aten_r1power_fr23:43:321
179950,0migration/128477-21latency21:58:371
165579955,0cyclictest0-21swapper/223:25:522
165579955,0cyclictest0-21swapper/200:00:042
165579954,0cyclictest24045-21sed20:58:362
165579950,0cyclictest0-21swapper/221:03:392
165569955,0cyclictest14818-21diskmemload22:47:371
165569955,0cyclictest14818-21diskmemload22:38:101
165569955,0cyclictest0-21swapper/121:38:341
165569950,0cyclictest0-21swapper/121:53:521
165549955,0cyclictest0-21swapper/023:04:140
165549950,0cyclictest0-21swapper/022:17:060
149950,0migration/08385-21kthreadcore00:08:360
149950,0migration/07849-21fschecks_count21:28:320
149950,0migration/07349-21irqcore23:28:340
149950,0migration/05605-21latency_hist00:08:190
149950,0migration/05582-21taskset21:44:470
149950,0migration/05000-21sendmail_mailqu21:23:420
149950,0migration/031533-21fschecks_count22:23:330
149950,0migration/030283-21kthreadcore20:38:360
149950,0migration/030226-21latency_hist23:43:190
149950,0migration/028030-21kthreadcore22:18:370
149950,0migration/027144-21gltestperf20:13:330
149950,0migration/026489-21sh23:58:370
149950,0migration/026235-21kthreadcore18:58:370
149950,0migration/024849-21kthreadcore21:38:370
149950,0migration/023551-21munin-node21:58:320
149950,0migration/022383-21taskset22:37:460
149950,0migration/022321-21taskset21:15:530
149950,0migration/018958-21kthreadcore18:53:360
149950,0migration/015934-21hwlatdetect21:53:340
149950,0migration/015201-21irqcore20:28:350
149950,0migration/014433-21sh23:13:250
149950,0migration/013123-21unixbench-2d21:08:420
339940,0migration/37750-21kthreadcore19:08:383
339940,0migration/37750-21kthreadcore19:08:383
339940,0migration/34355-21diskmemload23:44:213
339940,0migration/332595-21kernelversion21:03:363
339940,0migration/331223-21kthreadcore20:38:373
339940,0migration/330713-21fschecks_time19:03:333
339940,0migration/327768-21kthreadcore21:58:373
339940,0migration/326325-21kthreadcore21:18:373
339940,0migration/322765-21kthreadcore20:33:363
339940,0migration/317282-21fschecks_count18:53:323
339940,0migration/31224-21kthreadcore00:03:373
29725040,0irq/27-eth0-tx-6978-21cpuspeed_turbos22:28:322
29725040,0irq/27-eth0-tx-6032-21munin-node23:48:342
29725040,0irq/27-eth0-tx-31084-21ssh22:03:222
29725040,0irq/27-eth0-tx-15307-21ssh21:33:292
29725040,0irq/27-eth0-tx-13111-21munin-node20:03:320
29725040,0irq/27-eth0-tx-0-21swapper/223:34:112
29725040,0irq/27-eth0-tx-0-21swapper/223:04:232
29725040,0irq/27-eth0-tx-0-21swapper/222:43:512
29725040,0irq/27-eth0-tx-0-21swapper/222:18:302
29725040,0irq/27-eth0-tx-0-21swapper/222:13:342
29725040,0irq/27-eth0-tx-0-21swapper/221:38:302
29725040,0irq/27-eth0-tx-0-21swapper/221:13:222
29725040,0irq/27-eth0-tx-0-21swapper/220:54:072
29725040,0irq/27-eth0-tx-0-21swapper/200:19:582
29725040,0irq/27-eth0-tx-0-21swapper/020:18:430
29715040,0irq/26-eth0-rx-21090-21diskmemload22:34:053
29715040,0irq/26-eth0-rx-18066-21kthreadcore20:03:383
29715040,0irq/26-eth0-rx-0-21swapper/323:53:353
29715040,0irq/26-eth0-rx-0-21swapper/323:40:253
29715040,0irq/26-eth0-rx-0-21swapper/323:33:273
29715040,0irq/26-eth0-rx-0-21swapper/322:24:173
29715040,0irq/26-eth0-rx-0-21swapper/300:13:383
259940,0migration/27555-21aten_r1power_po20:23:322
259940,0migration/26635-21kthreadcore19:08:382
259940,0migration/26635-21kthreadcore19:08:382
259940,0migration/230174-21switchtime19:23:432
259940,0migration/228800-21needreboot18:58:382
259940,0migration/228171-21kthreadcore21:18:372
259940,0migration/219478-21kthreadcore18:53:362
259940,0migration/218847-21aten_r1power_en19:43:322
259940,0migration/216243-21kthreadcore19:13:372
259940,0migration/215205-21kthreadcore19:38:372
259940,0migration/212409-21unixbench_singl19:58:432
259940,0migration/210469-21kthreadcore21:08:362
249940,0watchdog/26340-21munin-node19:33:342
249940,0watchdog/232225-21tune2fs21:23:342
249940,0watchdog/225558-21kthreadcore22:38:372
249940,0watchdog/214746-21ssh22:33:252
249940,0watchdog/20-21swapper/220:08:382
249940,0watchdog/20-21swapper/219:03:342
179940,0migration/18947-21kthreadcore22:48:371
179940,0migration/16443-21kthreadcore20:43:371
179940,0migration/13270-21kthreadcore21:23:381
179940,0migration/130665-21kthreadcore20:13:371
179940,0migration/129753-21latency19:23:381
179940,0migration/129464-21taskset18:58:541
179940,0migration/127916-21needreboot23:18:371
179940,0migration/126312-21idleruntime19:48:341
179940,0migration/125239-21kthreadcore21:18:361
179940,0migration/121056-21kthreadcore18:53:361
179940,0migration/118102-21kthreadcore21:13:361
179940,0migration/114580-21fschecks_time00:13:331
179940,0migration/114126-21latency_hist23:33:191
179940,0migration/113590-21taskset20:25:201
165579943,0cyclictest3514-21snmpd20:50:122
165579943,0cyclictest3514-21snmpd20:38:272
165579943,0cyclictest3514-21snmpd20:28:272
165579943,0cyclictest3514-21snmpd20:19:272
165579943,0cyclictest3514-21snmpd20:14:572
165579943,0cyclictest3514-21snmpd20:04:572
165579943,0cyclictest3514-21snmpd19:28:272
165579943,0cyclictest3514-21snmpd19:20:572
165579943,0cyclictest31243-21awk23:43:402
165579943,0cyclictest2527-21awk20:43:342
165579943,0cyclictest23630-21awk22:58:322
165579943,0cyclictest14755-21awk00:13:342
165569943,0cyclictest7811-21awk22:08:351
165569943,0cyclictest7732-21awk22:28:351
165569943,0cyclictest7434-21awk23:08:341
165569943,0cyclictest7197-21awk23:28:341
165569943,0cyclictest3556-21lldpd21:33:341
165569943,0cyclictest3514-21snmpd23:53:571
165569943,0cyclictest3514-21snmpd23:49:571
165569943,0cyclictest3514-21snmpd23:14:571
165569943,0cyclictest3514-21snmpd23:03:571
165569943,0cyclictest3514-21snmpd22:22:571
165569943,0cyclictest3514-21snmpd22:04:121
165569943,0cyclictest3514-21snmpd21:03:271
165569943,0cyclictest3514-21snmpd21:00:571
165569943,0cyclictest3514-21snmpd20:55:571
165569943,0cyclictest3514-21snmpd20:48:271
165569943,0cyclictest3514-21snmpd00:02:421
165569943,0cyclictest32317-21awk21:43:341
165569943,0cyclictest23848-21awk22:38:361
165569943,0cyclictest22799-21awk00:18:341
165569943,0cyclictest15898-21awk22:13:341
165569943,0cyclictest12068-21awk19:13:351
165569943,0cyclictest10907-21sadc19:13:201
165569943,0cyclictest10907-21sadc19:13:201
165569940,0cyclictest0-21swapper/123:01:301
165549944,0cyclictest0-21swapper/023:23:570
165549944,0cyclictest0-21swapper/022:59:300
165549944,0cyclictest0-21swapper/022:54:470
165549944,0cyclictest0-21swapper/000:18:490
165549943,0cyclictest3556-21lldpd21:19:040
165549943,0cyclictest3556-21lldpd19:17:040
165549943,0cyclictest3514-21snmpd20:00:570
165549943,0cyclictest3514-21snmpd19:19:420
165549943,0cyclictest32150-21awk21:03:340
165549943,0cyclictest2603-21awk20:43:350
165549943,0cyclictest16218-21awk21:33:340
165549943,0cyclictest16105-21kworker/0:121:48:370
165549943,0cyclictest14771-21awk00:13:340
165549943,0cyclictest13140-21awk19:38:350
169940,0watchdog/16800-21chrt20:18:501
169940,0watchdog/14346-21rm00:04:531
169940,0watchdog/12619-21grep23:23:371
169940,0watchdog/12564-21kthreadcore22:23:371
169940,0watchdog/123640-21kthreadcore20:08:381
169940,0watchdog/123539-21irqrtprio23:38:371
169940,0watchdog/118741-21kthreadcore22:53:371
169940,0watchdog/10-21swapper/121:48:381
169940,0watchdog/10-21swapper/120:28:381
169940,0watchdog/10-21swapper/119:38:421
169940,0watchdog/10-21swapper/119:38:061
149940,0migration/07199-21fschecks_count22:28:330
149940,0migration/024127-21kthreadcore23:18:350
149940,0migration/022229-21kthreadcore20:33:360
149940,0migration/013576-21sh23:52:380
149940,0migration/011047-21kthreadcore22:08:380
339930,0migration/331264-21sh19:28:313
29715030,0irq/26-eth0-rx-0-21swapper/320:44:033
29715030,0irq/26-eth0-rx-0-21swapper/319:58:553
29715030,0irq/26-eth0-rx-0-21swapper/319:53:413
29715030,0irq/26-eth0-rx-0-21swapper/319:43:413
29715030,0irq/26-eth0-rx-0-21swapper/319:38:313
29715030,0irq/26-eth0-rx-0-21swapper/319:21:013
29715030,0irq/26-eth0-rx-0-21swapper/318:58:323
179930,0migration/123742-21kthreadcore19:43:401
165579932,0cyclictest3514-21snmpd19:55:272
165569932,0cyclictest605-21awk19:53:341
165569932,0cyclictest3514-21snmpd00:08:271
165549933,0cyclictest16105-21kworker/0:122:43:210
165549933,0cyclictest16105-21kworker/0:122:38:290
165549933,0cyclictest16105-21kworker/0:122:03:250
165549933,0cyclictest14661-21kworker/0:020:48:270
165549933,0cyclictest14661-21kworker/0:020:23:270
165549933,0cyclictest14661-21kworker/0:019:53:290
165549933,0cyclictest14661-21kworker/0:019:28:290
165549933,0cyclictest14661-21kworker/0:019:23:230
165549933,0cyclictest14661-21kworker/0:019:03:250
165549932,0cyclictest3241-21irqbalance23:08:270
169930,0watchdog/10-21swapper/121:31:381
169930,0watchdog/10-21swapper/121:09:341
169930,0watchdog/10-21swapper/120:03:381
169930,0watchdog/10-21swapper/119:28:301
169930,0watchdog/10-21swapper/119:03:341
169920,0watchdog/10-21swapper/120:38:221
169920,0watchdog/10-21swapper/120:33:221
169920,0watchdog/10-21swapper/119:58:221
169920,0watchdog/10-21swapper/119:18:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional