You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-27 - 16:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sat Jun 27, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351670,0ktimersoftd/312441-21perf06:51:183
1898024637,0sleep20-21swapper/206:51:152
1864523624,0sleep00-21swapper/006:47:200
1866823127,0sleep10-21swapper/106:47:331
339960,0migration/37586-21sh11:23:213
339960,0migration/31996-21munin-node10:21:413
339960,0migration/315608-21sh11:07:593
29715060,0irq/26-eth0-rx-7870-21ps12:04:293
29715060,0irq/26-eth0-rx-639-21munin-run11:01:273
29715060,0irq/26-eth0-rx-30898-21latency10:16:463
149960,0migration/07622-21timerwakeupswit09:21:500
149960,0migration/015245-21sh11:47:230
149960,0migration/010907-21kthreadcore10:06:460
339950,0migration/39831-21irqcore07:56:443
339950,0migration/35963-21kthreadcore08:41:443
339950,0migration/331756-21taskset11:38:173
339950,0migration/330958-21kthreadcore07:21:463
339950,0migration/328780-21kthreadcore11:56:453
339950,0migration/325998-21aten_r1power_fr06:56:403
339950,0migration/325575-21cpuspeed_turbos08:56:403
339950,0migration/325196-21cpuspeed_turbos12:16:403
339950,0migration/324160-21taskset10:13:143
339950,0migration/322657-21sendmail_mailst12:11:503
339950,0migration/320708-21munin-node08:01:463
339950,0migration/318723-21munin-node08:51:463
339950,0migration/318067-21munin-node11:31:413
339950,0migration/317641-21sh09:51:283
339950,0migration/317447-21aten_r1power_vo10:51:403
339950,0migration/31713-21munin-node09:41:393
339950,0migration/316038-21sh10:28:263
339950,0migration/314128-21kthreadcore09:46:453
339950,0migration/313974-21kthreadcore09:06:463
29725050,0irq/27-eth0-tx-9518-21munin-node10:06:472
29725050,0irq/27-eth0-tx-8929-21munin-node11:46:402
29725050,0irq/27-eth0-tx-25300-21munin-node08:56:432
29725050,0irq/27-eth0-tx-24938-21munin-node12:16:502
29725050,0irq/27-eth0-tx-22112-21munin-node08:06:410
29725050,0irq/27-eth0-tx-16683-21ssh09:09:512
29725050,0irq/27-eth0-tx-16061-21ssh10:28:292
29725050,0irq/27-eth0-tx-15216-21ntp_offset09:46:472
29725050,0irq/27-eth0-tx-0-21swapper/212:04:252
29725050,0irq/27-eth0-tx-0-21swapper/209:41:442
29725050,0irq/27-eth0-tx-0-21swapper/208:55:432
29725050,0irq/27-eth0-tx-0-21swapper/008:53:310
29725050,0irq/27-eth0-tx-0-21swapper/008:01:390
29725050,0irq/27-eth0-tx-0-21swapper/007:36:400
29725050,0irq/27-eth0-tx-0-21swapper/007:13:520
29715050,0irq/26-eth0-rx-32684-21cat08:36:443
29715050,0irq/26-eth0-rx-32132-21seq09:37:133
29715050,0irq/26-eth0-rx-18327-21tune2fs09:31:423
29715050,0irq/26-eth0-rx-1614-21tune2fs11:01:433
29715050,0irq/26-eth0-rx-12147-21kthreadcore09:26:453
29715050,0irq/26-eth0-rx-11616-21if_err_eth1.208:46:433
29715050,0irq/26-eth0-rx-0-21swapper/311:53:453
29715050,0irq/26-eth0-rx-0-21swapper/311:51:093
29715050,0irq/26-eth0-rx-0-21swapper/311:46:003
29715050,0irq/26-eth0-rx-0-21swapper/311:30:113
29715050,0irq/26-eth0-rx-0-21swapper/311:11:463
29715050,0irq/26-eth0-rx-0-21swapper/310:40:003
29715050,0irq/26-eth0-rx-0-21swapper/310:06:463
29715050,0irq/26-eth0-rx-0-21swapper/310:05:583
29715050,0irq/26-eth0-rx-0-21swapper/309:21:453
29715050,0irq/26-eth0-rx-0-21swapper/309:11:583
29715050,0irq/26-eth0-rx-0-21swapper/307:51:413
259950,0migration/2938-21sh11:01:352
259950,0migration/28705-21latency_hist12:06:272
259950,0migration/226567-21fschecks_time06:56:422
259950,0migration/226218-21irqcore11:36:442
259950,0migration/225905-21sh09:16:342
259950,0migration/224756-21sh11:36:182
259950,0migration/222942-21timerwakeupswit11:51:502
259950,0migration/216847-21kthreadcore08:01:442
259950,0migration/216799-21latency_hist11:31:272
259950,0migration/21663-21fschecks_count09:01:422
249950,0watchdog/226560-21netstat07:41:462
190759955,0cyclictest908-21crond10:21:272
190759954,0cyclictest0-21swapper/206:51:482
190759950,0cyclictest0-21swapper/211:13:592
190749955,0cyclictest0-21swapper/112:18:451
190749955,0cyclictest0-21swapper/111:49:161
190749955,0cyclictest0-21swapper/109:26:071
190749954,0cyclictest3514-21snmpd12:16:011
190749950,0cyclictest0-21swapper/110:20:401
190749950,0cyclictest0-21swapper/110:13:441
190749950,0cyclictest0-21swapper/109:57:301
190749950,0cyclictest0-21swapper/109:09:341
190749950,0cyclictest0-21swapper/108:46:521
19150,0ktimersoftd/10-21swapper/111:04:391
179950,0migration/123890-21sh09:51:571
169950,0watchdog/130952-21cut08:11:461
169950,0watchdog/116999-21grep08:26:421
169950,0watchdog/10-21swapper/108:41:501
149950,0migration/0678-21taskset11:41:340
149950,0migration/05293-21kthreadcore12:01:430
149950,0migration/0427-21sh09:19:080
149950,0migration/03719-21kthreadcore07:26:440
149950,0migration/03497-21kthreadcore07:51:450
149950,0migration/032017-21sh09:57:160
149950,0migration/030430-21kthreadcore10:36:460
149950,0migration/029667-21kthreadcore10:56:440
149950,0migration/029257-21kthreadcore07:21:450
149950,0migration/026251-21taskset07:17:190
149950,0migration/024864-21kthreadcore08:31:460
149950,0migration/024690-21taskset09:34:030
149950,0migration/024512-21kthreadcore07:41:440
149950,0migration/022788-21taskset08:30:450
149950,0migration/02065-21sh10:41:430
149950,0migration/020295-21kthreadcore10:51:450
149950,0migration/020001-21kthreadcore11:11:460
149950,0migration/017526-21latency_hist09:51:270
149950,0migration/016957-21sh11:51:370
149950,0migration/016267-21ksmtuned12:10:290
149950,0migration/013643-21kthreadcore10:26:450
149950,0migration/013121-21taskset07:09:100
149950,0migration/01275-21cpuspeed_turbos11:01:410
149950,0migration/012633-21kthreadcore09:26:450
149950,0migration/010440-21kthreadcore07:56:450
4140,0ktimersoftd/00-21swapper/011:57:110
339940,0migration/39859-21fschecks_time10:46:423
339940,0migration/37859-21if_eth1.207:06:433
339940,0migration/3711-21sh09:19:573
339940,0migration/331010-21swap11:16:503
339940,0migration/329342-21kthreadcore09:56:453
339940,0migration/32669-21kthreadcore07:01:453
339940,0migration/323995-21kthreadcore08:06:453
339940,0migration/319182-21kthreadcore08:26:463
339940,0migration/314901-21kthreadcore07:11:443
339940,0migration/310405-21kthreadcore07:31:453
29725040,0irq/27-eth0-tx-17390-21munin-node10:31:412
29725040,0irq/27-eth0-tx-0-21swapper/212:11:452
29725040,0irq/27-eth0-tx-0-21swapper/210:56:282
29725040,0irq/27-eth0-tx-0-21swapper/210:51:312
29725040,0irq/27-eth0-tx-0-21swapper/210:36:402
29725040,0irq/27-eth0-tx-0-21swapper/210:21:362
29725040,0irq/27-eth0-tx-0-21swapper/210:01:442
29725040,0irq/27-eth0-tx-0-21swapper/209:36:372
29725040,0irq/27-eth0-tx-0-21swapper/209:31:352
29725040,0irq/27-eth0-tx-0-21swapper/209:26:462
29725040,0irq/27-eth0-tx-0-21swapper/008:46:390
29715040,0irq/26-eth0-rx-23637-21ssh10:32:033
29715040,0irq/26-eth0-rx-1226-21sh10:41:393
29715040,0irq/26-eth0-rx-0-21swapper/312:06:433
29715040,0irq/26-eth0-rx-0-21swapper/309:02:293
29715040,0irq/26-eth0-rx-0-21swapper/308:21:463
29715040,0irq/26-eth0-rx-0-21swapper/307:46:453
29715040,0irq/26-eth0-rx-0-21swapper/307:26:503
29715040,0irq/26-eth0-rx-0-21swapper/306:51:423
259940,0migration/29906-21munin-node11:06:432
259940,0migration/2551-21kthreadcore08:36:442
259940,0migration/25376-21kthreadcore07:26:452
259940,0migration/24033-21kthreadcore11:21:452
259940,0migration/230918-21kthreadcore07:46:452
259940,0migration/227116-21kthreadcore11:16:442
259940,0migration/226040-21cpuspeed_turbos09:56:402
259940,0migration/223052-21latency_hist08:31:282
259940,0migration/221966-21kthreadcore09:51:442
259940,0migration/220876-21kthreadcore08:26:462
259940,0migration/217712-21aten_r1power_po09:11:382
259940,0migration/215570-21timerandwakeup08:21:512
259940,0migration/215018-21kthreadcore07:11:442
259940,0migration/211933-21irqrtprio08:46:442
259940,0migration/211902-21kthreadcore07:56:452
249940,0watchdog/29806-21switchtime08:41:502
249940,0watchdog/215115-21fschecks_time07:36:422
249940,0watchdog/20-21swapper/210:46:452
249940,0watchdog/20-21swapper/210:41:462
249940,0watchdog/20-21swapper/210:12:582
190759944,0cyclictest0-21swapper/208:06:412
190759943,0cyclictest8700-21awk07:31:432
190759943,0cyclictest680-21ssh11:41:342
190759943,0cyclictest3556-21lldpd08:18:012
190759943,0cyclictest3556-21lldpd07:21:302
190759943,0cyclictest26359-21kthreadcore11:56:442
190759943,0cyclictest2518-21awk09:21:422
190759940,0cyclictest0-21swapper/207:51:422
190749944,0cyclictest0-21swapper/111:42:581
190749943,0cyclictest972-21awk07:01:421
190749943,0cyclictest3556-21lldpd10:29:011
190749943,0cyclictest3556-21lldpd10:21:311
190749943,0cyclictest3556-21lldpd08:56:311
190749943,0cyclictest3514-21snmpd11:56:461
190749943,0cyclictest3514-21snmpd07:46:001
190749943,0cyclictest3514-21snmpd07:33:001
190749943,0cyclictest3241-21irqbalance11:17:261
190749943,0cyclictest3241-21irqbalance07:07:471
190749943,0cyclictest26381-21awk10:56:421
190749943,0cyclictest26346-21awk10:36:431
190749943,0cyclictest2048-21awk07:26:431
190749943,0cyclictest14417-21awk07:11:421
190749943,0cyclictest10557-21awk09:26:431
190749943,0cyclictest10174-21awk10:46:431
190749943,0cyclictest10039-21awk11:06:441
190739944,0cyclictest0-21swapper/010:24:090
190739944,0cyclictest0-21swapper/010:14:510
190739944,0cyclictest0-21swapper/010:03:110
190739944,0cyclictest0-21swapper/009:00:070
190739943,0cyclictest4074-21awk08:16:450
190739943,0cyclictest25986-21awk11:36:420
190739943,0cyclictest2132-21awk09:01:440
190739942,0cyclictest29544-21kworker/0:209:10:310
190739940,0cyclictest0-21swapper/009:52:280
179940,0migration/132033-21taskset06:58:231
179940,0migration/123544-21kthreadcore07:16:461
179940,0migration/11816-21munin-node12:01:421
179940,0migration/117966-21fschecks_count08:51:431
179940,0migration/117685-21munin-node09:11:381
179940,0migration/11532-21aten_r1power_en09:41:381
179940,0migration/110039-21irqcore11:26:441
169940,0watchdog/19407-21munin-node12:06:411
169940,0watchdog/15860-21cut11:21:451
169940,0watchdog/130571-21cut09:16:461
169940,0watchdog/123602-21ssh10:31:581
169940,0watchdog/122206-21kthreadcore11:51:451
169940,0watchdog/120301-21irqcore06:51:461
169940,0watchdog/118402-21perl09:31:421
169940,0watchdog/10-21swapper/110:51:451
169940,0watchdog/10-21swapper/109:39:461
169940,0watchdog/10-21swapper/109:01:461
169940,0watchdog/10-21swapper/107:36:461
149940,0migration/09808-21idleruntime11:26:420
149940,0migration/030393-21missed_timers12:16:470
149940,0migration/023827-21runrttasks09:11:570
149940,0migration/019329-21kthreadcore10:31:450
149940,0migration/011901-21kthreadcore07:31:450
35130,0ktimersoftd/30-21swapper/308:17:163
29725030,0irq/27-eth0-tx-0-21swapper/008:11:420
29725030,0irq/27-eth0-tx-0-21swapper/007:46:430
29725030,0irq/27-eth0-tx-0-21swapper/006:51:420
29715030,0irq/26-eth0-rx-0-21swapper/308:31:413
29715030,0irq/26-eth0-rx-0-21swapper/308:11:523
29715030,0irq/26-eth0-rx-0-21swapper/307:41:393
29715030,0irq/26-eth0-rx-0-21swapper/307:36:403
29715030,0irq/26-eth0-rx-0-21swapper/307:16:523
249930,0watchdog/20-21swapper/207:16:462
190759932,0cyclictest29606-21awk08:11:442
190749933,0cyclictest3556-21lldpd10:02:011
190749932,0cyclictest3845-21head08:16:431
190749932,0cyclictest3514-21snmpd08:39:161
190749932,0cyclictest3241-21irqbalance08:31:361
190749932,0cyclictest29196-21kthreadcore07:46:451
190749932,0cyclictest23126-21irqcore08:06:441
190749932,0cyclictest1991-21awk10:41:421
190749932,0cyclictest18086-21awk11:11:441
190749932,0cyclictest17187-21diskmemload11:31:311
190749932,0cyclictest13717-21cut09:46:451
190749932,0cyclictest12591-21grep07:56:451
190739933,0cyclictest29544-21kworker/0:208:37:310
190739933,0cyclictest1248-21kworker/0:012:11:460
190739933,0cyclictest1248-21kworker/0:011:32:020
190739933,0cyclictest1248-21kworker/0:011:21:430
190739933,0cyclictest1248-21kworker/0:011:16:530
190739933,0cyclictest1248-21kworker/0:010:16:290
190739933,0cyclictest1248-21kworker/0:009:41:290
190739933,0cyclictest1248-21kworker/0:009:36:410
190739933,0cyclictest12402-21kworker/0:107:01:300
190739933,0cyclictest0-21swapper/010:46:320
190739932,0cyclictest4924-21awk08:41:420
190739932,0cyclictest3556-21lldpd11:06:310
190739932,0cyclictest25919-21munin-node06:56:410
190739932,0cyclictest10624-21awk08:21:430
169930,0watchdog/117187-21diskmemload11:36:451
169930,0watchdog/10-21swapper/110:06:341
169930,0watchdog/10-21swapper/108:21:461
169930,0watchdog/10-21swapper/108:01:421
169930,0watchdog/10-21swapper/107:51:461
169930,0watchdog/10-21swapper/107:21:461
249920,0watchdog/20-21swapper/207:06:302
190759921,0cyclictest0-21swapper/207:01:282
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional