You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 16:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Mon Dec 29, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450760,0irq/25-eth10-21swapper/307:04:543
609323829,0sleep20-21swapper/207:06:082
617023725,0sleep10-21swapper/107:06:551
602023624,0sleep00-21swapper/007:05:180
63819976,0cyclictest7947-21turbostat.cron12:03:272
63819960,0cyclictest4678-21diskmemload11:26:492
63819960,0cyclictest0-21swapper/212:28:092
63819960,0cyclictest0-21swapper/211:53:282
63799965,0cyclictest29537-21ssh09:21:590
35160,0ktimersoftd/30-21swapper/311:41:043
339960,0migration/39197-21sh10:25:233
339960,0migration/37305-21sh12:01:543
339960,0migration/325577-21sh10:17:233
339960,0migration/323417-21sh09:54:513
339960,0migration/322020-21sh11:49:493
339960,0migration/32183-21sh10:41:063
339960,0migration/316073-21sh10:09:333
29715060,0irq/26-eth0-rx-3514-21snmpd10:51:183
29715060,0irq/26-eth0-rx-26413-21ps09:58:443
29715060,0irq/26-eth0-rx-25840-21ntp_kernel_err10:53:493
29715060,0irq/26-eth0-rx-24530-21munin-node10:33:453
29715060,0irq/26-eth0-rx-1743-21perl10:03:413
29715060,0irq/26-eth0-rx-11088-21users09:08:513
29715060,0irq/26-eth0-rx-0-21swapper/311:28:453
169960,0watchdog/16488-21tune2fs11:23:411
63819955,0cyclictest0-21swapper/211:15:572
63819955,0cyclictest0-21swapper/208:38:552
63819954,0cyclictest7876-21tr11:03:432
63819950,0cyclictest0-21swapper/210:51:022
63819950,0cyclictest0-21swapper/210:40:092
63819950,0cyclictest0-21swapper/210:00:542
63819950,0cyclictest0-21swapper/209:41:302
63809955,0cyclictest0-21swapper/109:16:591
63809954,0cyclictest18741-21ssh10:50:281
63809950,0cyclictest0-21swapper/110:39:071
63799955,0cyclictest0-21swapper/010:00:180
63799955,0cyclictest0-21swapper/007:38:520
63799955,0cyclictest0-21swapper/007:28:280
63799950,0cyclictest0-21swapper/009:35:130
4150,0ktimersoftd/00-21swapper/010:56:160
339950,0migration/36488-21missed_timers12:18:463
339950,0migration/36105-21kthreadcore07:28:463
339950,0migration/33338-21munin-node11:18:503
339950,0migration/330598-21kthreadcore10:58:473
339950,0migration/329856-21kthreadcore10:18:433
339950,0migration/329548-21munin-node11:53:483
339950,0migration/328831-21kthreadcore12:13:463
339950,0migration/328111-21sh09:19:013
339950,0migration/327886-21kthreadcore08:33:463
339950,0migration/327114-21munin-node11:13:463
339950,0migration/323904-21latency_hist09:38:283
339950,0migration/317887-21aten_r1power_vo12:28:393
339950,0migration/315384-21munin-node08:23:503
339950,0migration/314499-21taskset09:32:083
339950,0migration/312804-21taskset07:10:373
339950,0migration/311399-21idleruntime08:48:423
29725050,0irq/27-eth0-tx-7665-21munin-node07:33:492
29725050,0irq/27-eth0-tx-6600-21munin-node07:08:412
29725050,0irq/27-eth0-tx-6168-21unixbench_singl09:43:502
29725050,0irq/27-eth0-tx-3265-21sshd11:00:482
29725050,0irq/27-eth0-tx-29088-21cut12:13:462
29725050,0irq/27-eth0-tx-17739-21chrt12:28:372
29725050,0irq/27-eth0-tx-16860-21munin-node09:53:402
29725050,0irq/27-eth0-tx-0-21swapper/211:28:582
29725050,0irq/27-eth0-tx-0-21swapper/209:33:452
29725050,0irq/27-eth0-tx-0-21swapper/209:18:002
29715050,0irq/26-eth0-rx-7144-21ssh09:45:563
29715050,0irq/26-eth0-rx-451-21ssh12:35:233
29715050,0irq/26-eth0-rx-31798-21kthreadcore08:38:473
29715050,0irq/26-eth0-rx-18918-21ssh09:13:493
29715050,0irq/26-eth0-rx-0-21swapper/312:23:493
29715050,0irq/26-eth0-rx-0-21swapper/312:10:563
29715050,0irq/26-eth0-rx-0-21swapper/312:03:293
29715050,0irq/26-eth0-rx-0-21swapper/311:43:373
29715050,0irq/26-eth0-rx-0-21swapper/311:24:233
29715050,0irq/26-eth0-rx-0-21swapper/311:10:283
29715050,0irq/26-eth0-rx-0-21swapper/310:29:493
29715050,0irq/26-eth0-rx-0-21swapper/309:35:103
29715050,0irq/26-eth0-rx-0-21swapper/309:27:003
29715050,0irq/26-eth0-rx-0-21swapper/308:43:503
29715050,0irq/26-eth0-rx-0-21swapper/307:33:383
27150,0ktimersoftd/20-21swapper/210:10:592
27150,0ktimersoftd/20-21swapper/208:38:272
259950,0migration/26583-21ntp_kernel_pll_12:18:472
259950,0migration/232311-21kthreadcore09:03:432
259950,0migration/228419-21idleruntime07:48:422
259950,0migration/224638-21kthreadcore07:43:452
259950,0migration/223469-21munin-node11:33:422
259950,0migration/221684-21kthreadcore10:53:462
259950,0migration/214000-21gltestperf07:13:422
259950,0migration/213757-21sh09:30:352
249950,0watchdog/20-21swapper/210:13:462
179950,0migration/19777-21sh12:23:381
179950,0migration/16200-21sh11:59:201
179950,0migration/14672-21sh09:24:311
179950,0migration/14090-21timerwakeupswit09:03:501
179950,0migration/128438-21kthreadcore10:18:421
179950,0migration/126162-21taskset07:20:571
179950,0migration/124629-21needreboot10:33:461
179950,0migration/123095-21fschecks_count11:33:411
179950,0migration/121908-21fschecks_count11:13:391
179950,0migration/115198-21kthreadcore11:08:441
179950,0migration/113059-21latency_hist09:13:281
149950,0migration/06723-21sh09:44:570
149950,0migration/05557-21sh09:26:310
149950,0migration/05274-21fschecks_time11:03:400
149950,0migration/032275-21kthreadcore09:03:430
149950,0migration/031029-21irqrtprio11:18:430
149950,0migration/028517-21timerandwakeup11:33:500
149950,0migration/027523-21if_eth007:23:430
149950,0migration/024615-21kthreadcore07:18:460
149950,0migration/024491-21sh10:14:450
149950,0migration/023171-21sh12:09:500
149950,0migration/022128-21munin-node08:08:410
149950,0migration/019837-21kthreadcore09:53:440
149950,0migration/019573-21kthreadcore12:28:430
149950,0migration/017679-21sendmail_mailtr10:48:470
149950,0migration/013861-21sh09:30:470
149950,0migration/013850-21taskset07:36:090
149950,0migration/013825-21ntp_offset12:03:460
149950,0migration/013472-21cpuspeed_turbos09:13:400
149950,0migration/01082-21taskset12:36:460
63819943,0cyclictest3556-21lldpd08:00:502
63819943,0cyclictest3514-21snmpd08:59:022
63819943,0cyclictest2787-21awk07:53:442
63819943,0cyclictest27527-21awk07:23:432
63809944,0cyclictest0-21swapper/108:38:501
63809943,0cyclictest7469-21awk07:08:431
63809943,0cyclictest4774-21awk08:43:421
63809943,0cyclictest3556-21lldpd12:33:531
63809943,0cyclictest3556-21lldpd12:06:541
63809943,0cyclictest3556-21lldpd09:55:221
63809943,0cyclictest3556-21lldpd09:01:511
63809943,0cyclictest3556-21lldpd08:08:211
63809943,0cyclictest3556-21lldpd07:54:201
63809943,0cyclictest3514-21snmpd08:28:471
63809943,0cyclictest25938-21awk12:13:441
63809943,0cyclictest22704-21awk09:18:421
63809943,0cyclictest21202-21awk10:53:441
63809943,0cyclictest18890-21awk10:13:431
63809943,0cyclictest16397-21awk11:48:441
63809943,0cyclictest14310-21irqcore07:13:441
63809940,0cyclictest0-21swapper/112:09:081
63799943,0cyclictest4759-21awk08:43:420
63799943,0cyclictest3556-21lldpd12:27:240
63799943,0cyclictest3514-21snmpd10:28:170
63799943,0cyclictest3514-21snmpd10:11:020
63799943,0cyclictest3514-21snmpd07:14:020
63799943,0cyclictest3241-21irqbalance09:48:460
63799943,0cyclictest28530-21awk07:48:430
63799943,0cyclictest21901-21awk07:43:430
63799943,0cyclictest10478-21awk08:23:420
339940,0migration/34533-21taskset09:07:023
339940,0migration/33431-21fschecks_count08:18:413
339940,0migration/33149-21irqrtprio07:53:453
339940,0migration/326241-21kthreadcore08:58:453
339940,0migration/312816-21kthreadcore07:58:453
29725040,0irq/27-eth0-tx-31101-21kthreadcore12:33:452
29725040,0irq/27-eth0-tx-28196-21ssh09:19:152
29725040,0irq/27-eth0-tx-0-21swapper/212:04:392
29725040,0irq/27-eth0-tx-0-21swapper/211:43:352
29725040,0irq/27-eth0-tx-0-21swapper/211:38:422
29725040,0irq/27-eth0-tx-0-21swapper/211:18:552
29725040,0irq/27-eth0-tx-0-21swapper/210:43:322
29725040,0irq/27-eth0-tx-0-21swapper/210:33:362
29725040,0irq/27-eth0-tx-0-21swapper/210:28:492
29725040,0irq/27-eth0-tx-0-21swapper/210:23:452
29725040,0irq/27-eth0-tx-0-21swapper/210:03:392
29725040,0irq/27-eth0-tx-0-21swapper/209:48:282
29725040,0irq/27-eth0-tx-0-21swapper/209:23:502
29715040,0irq/26-eth0-rx-0-21swapper/311:34:223
29715040,0irq/26-eth0-rx-0-21swapper/311:04:263
29715040,0irq/26-eth0-rx-0-21swapper/310:43:453
29715040,0irq/26-eth0-rx-0-21swapper/309:48:283
29715040,0irq/26-eth0-rx-0-21swapper/307:27:373
29715040,0irq/26-eth0-rx-0-21swapper/307:18:463
259940,0migration/28653-21kthreadcore08:43:452
259940,0migration/26168-21kthreadcore08:18:452
259940,0migration/25031-21aten2.4-expect09:08:392
259940,0migration/23752-21kthreadcore07:28:452
259940,0migration/231513-21kthreadcore08:13:452
259940,0migration/230604-21kthreadcore10:18:432
259940,0migration/222540-21latency08:53:472
259940,0migration/221152-21kthreadcore08:28:482
259940,0migration/220341-21kthreadcore08:03:462
259940,0migration/220331-21kthreadcore12:08:452
249940,0watchdog/225037-21kthreadcore07:18:462
249940,0watchdog/215843-21diskstats11:48:412
249940,0watchdog/213105-21ssh11:08:372
249940,0watchdog/20-21swapper/208:48:502
249940,0watchdog/20-21swapper/207:38:462
179940,0migration/19735-21aten_r1power_po10:08:391
179940,0migration/15729-21kthreadcore08:18:441
179940,0migration/14324-21hwlatdetect10:43:421
179940,0migration/13400-21timerandwakeup11:18:501
179940,0migration/128803-21kthreadcore11:53:441
179940,0migration/127323-21taskset07:48:071
179940,0migration/125869-21fschecks_time09:58:421
179940,0migration/118671-21if_eth012:28:421
179940,0migration/116759-21kthreadcore11:28:451
179940,0migration/116173-21kthreadcore07:38:451
179940,0migration/114782-21diskmemload09:49:111
179940,0migration/113525-21diskmemload09:30:051
179940,0migration/112505-21kthreadcore10:28:461
179940,0migration/111846-21kthreadcore11:43:441
169940,0watchdog/18010-21ssh09:48:021
169940,0watchdog/15593-21cut10:23:461
169940,0watchdog/14664-21kthreadcore12:18:451
169940,0watchdog/1444-21runrttasks07:27:221
169940,0watchdog/13639-21ssh11:01:381
169940,0watchdog/127172-21cat09:38:461
169940,0watchdog/124123-21cat08:08:461
169940,0watchdog/122576-21ssh09:35:581
169940,0watchdog/113079-21ntp_kernel_pll_07:33:461
169940,0watchdog/112571-21ssh11:07:531
169940,0watchdog/10-21swapper/111:38:451
169940,0watchdog/10-21swapper/108:13:461
149940,0migration/09131-21sh10:07:590
149940,0migration/07514-21diskmemload12:02:240
149940,0migration/05716-21kthreadcore10:43:440
149940,0migration/030018-21taskset08:59:270
149940,0migration/029808-21kthreadcore12:13:460
149940,0migration/028893-21kthreadcore10:18:420
149940,0migration/022520-21irqcore11:13:420
149940,0migration/022213-21sh11:33:110
149940,0migration/020014-21runrttasks11:10:400
149940,0migration/016923-21kthreadcore11:48:450
149940,0migration/014332-21kthreadcore10:28:470
149940,0migration/013666-21sh11:27:460
149940,0migration/010793-21kthreadcore07:58:440
63819932,0cyclictest22317-21tr08:08:422
63819930,0cyclictest0-21swapper/208:23:502
63809933,0cyclictest3556-21lldpd07:31:201
63809933,0cyclictest11431-21awk08:48:421
63809932,0cyclictest3556-21lldpd10:05:521
63809932,0cyclictest29058-21grep07:48:441
63809932,0cyclictest25825-21cut08:33:461
63809932,0cyclictest18157-21awk08:53:441
63799933,0cyclictest3556-21lldpd11:39:230
63799933,0cyclictest14765-21kworker/0:009:08:450
63799933,0cyclictest0-21swapper/011:53:480
63799933,0cyclictest0-21swapper/010:40:490
63799933,0cyclictest0-21swapper/009:42:290
63799933,0cyclictest0-21swapper/008:03:380
63799932,0cyclictest7983-21awk11:43:420
63799932,0cyclictest3514-21snmpd12:21:540
63799932,0cyclictest3514-21snmpd07:12:010
63799932,0cyclictest29662-21sh08:13:440
63799932,0cyclictest2772-21awk07:53:430
63799932,0cyclictest20186-21grep10:33:430
63799932,0cyclictest18207-21awk08:53:440
63799932,0cyclictest16278-21sensors08:48:490
63799932,0cyclictest1427-21kthreadcore10:58:480
63799930,0cyclictest28612-21python08:33:500
339930,0migration/319258-21unixbench_singl07:13:513
29715030,0irq/26-eth0-rx-0-21swapper/308:56:143
29715030,0irq/26-eth0-rx-0-21swapper/308:33:113
29715030,0irq/26-eth0-rx-0-21swapper/308:13:413
29715030,0irq/26-eth0-rx-0-21swapper/308:08:413
29715030,0irq/26-eth0-rx-0-21swapper/308:06:463
29715030,0irq/26-eth0-rx-0-21swapper/307:49:513
29715030,0irq/26-eth0-rx-0-21swapper/307:38:583
169930,0watchdog/10-21swapper/108:23:421
169930,0watchdog/10-21swapper/107:58:421
63799921,0cyclictest3762-21crond08:18:280
63799920,0cyclictest16269-21latency_hist08:28:280
63799920,0cyclictest0-21swapper/008:38:280
29715020,0irq/26-eth0-rx-0-21swapper/307:43:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional