You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 01:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed Dec 24, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450650,0irq/25-eth10-21swapper/307:05:183
1077923728,0sleep10-21swapper/107:07:281
1074023627,0sleep20-21swapper/207:07:002
1054123423,0sleep00-21swapper/007:04:380
109789976,0cyclictest0-21swapper/012:19:130
35160,0ktimersoftd/30-21swapper/307:54:153
339960,0migration/36732-21munin-node10:39:183
339960,0migration/329253-21sh09:56:493
339960,0migration/311988-21munin-node11:59:133
29715060,0irq/26-eth0-rx-2951-21perl10:59:103
259960,0migration/27913-21munin-node10:04:112
19160,0ktimersoftd/111797-21missed_timers07:54:151
149960,0migration/09953-21sh11:19:250
149960,0migration/07414-21sh10:40:020
149960,0migration/031681-21munin-node11:54:130
149960,0migration/027257-21munin-node08:54:180
339950,0migration/39140-21munin-node09:24:203
339950,0migration/35707-21fschecks_count11:39:083
339950,0migration/35136-21kthreadcore11:19:123
339950,0migration/330809-21kthreadcore08:59:143
339950,0migration/330672-21sshd07:21:183
339950,0migration/329748-21kthreadcore08:09:143
339950,0migration/32770-21kthreadcore12:34:133
339950,0migration/327436-21latency_hist08:33:563
339950,0migration/325395-21fschecks_count12:29:093
339950,0migration/324744-21taskset10:51:023
339950,0migration/323132-21kthreadcore09:14:143
339950,0migration/323109-21kthreadcore08:04:133
339950,0migration/322654-21munin-node08:54:133
339950,0migration/322441-21kthreadcore11:29:143
339950,0migration/31975-21munin-node08:14:133
339950,0migration/317130-21fschecks_time12:24:103
339950,0migration/316033-21if_eth108:49:113
339950,0migration/315512-21taskset12:22:243
339950,0migration/315011-21sh12:04:023
339950,0migration/314899-21gltestperf08:24:103
339950,0migration/312402-21kthreadcore10:24:123
339950,0migration/310870-21aten_r1power_cu11:04:063
29725050,0irq/27-eth0-tx-7986-21munin-node10:24:142
29725050,0irq/27-eth0-tx-7737-21latency_hist12:18:552
29725050,0irq/27-eth0-tx-5448-21ssh10:02:002
29725050,0irq/27-eth0-tx-30857-21tr09:19:142
29725050,0irq/27-eth0-tx-29193-21find10:14:172
29725050,0irq/27-eth0-tx-27471-21munin-node08:34:112
29725050,0irq/27-eth0-tx-2680-21munin-node09:04:092
29725050,0irq/27-eth0-tx-20493-21ssh11:46:012
29725050,0irq/27-eth0-tx-14286-21fschecks_time09:49:102
29725050,0irq/27-eth0-tx-11302-21ssh09:44:212
29725050,0irq/27-eth0-tx-0-21swapper/212:34:502
29725050,0irq/27-eth0-tx-0-21swapper/212:17:112
29725050,0irq/27-eth0-tx-0-21swapper/211:51:412
29725050,0irq/27-eth0-tx-0-21swapper/211:30:242
29725050,0irq/27-eth0-tx-0-21swapper/211:01:522
29725050,0irq/27-eth0-tx-0-21swapper/210:53:592
29725050,0irq/27-eth0-tx-0-21swapper/210:32:492
29715050,0irq/26-eth0-rx-4364-21taskset11:37:453
29715050,0irq/26-eth0-rx-28496-21kthreadcore12:09:143
29715050,0irq/26-eth0-rx-23254-21ssh10:30:523
29715050,0irq/26-eth0-rx-17304-21taskset09:29:173
29715050,0irq/26-eth0-rx-17304-21taskset09:29:173
29715050,0irq/26-eth0-rx-16195-21ssh09:11:053
29715050,0irq/26-eth0-rx-136022chrt08:20:263
29715050,0irq/26-eth0-rx-0-21swapper/312:15:413
29715050,0irq/26-eth0-rx-0-21swapper/311:54:523
29715050,0irq/26-eth0-rx-0-21swapper/311:50:373
29715050,0irq/26-eth0-rx-0-21swapper/311:47:303
29715050,0irq/26-eth0-rx-0-21swapper/311:24:083
29715050,0irq/26-eth0-rx-0-21swapper/311:15:383
29715050,0irq/26-eth0-rx-0-21swapper/310:54:133
29715050,0irq/26-eth0-rx-0-21swapper/310:44:123
29715050,0irq/26-eth0-rx-0-21swapper/310:35:103
29715050,0irq/26-eth0-rx-0-21swapper/310:16:433
29715050,0irq/26-eth0-rx-0-21swapper/310:09:363
29715050,0irq/26-eth0-rx-0-21swapper/310:04:193
29715050,0irq/26-eth0-rx-0-21swapper/309:39:233
29715050,0irq/26-eth0-rx-0-21swapper/309:38:553
29715050,0irq/26-eth0-rx-0-21swapper/309:19:393
29715050,0irq/26-eth0-rx-0-21swapper/307:34:173
259950,0migration/24079-21timerandwakeup11:54:202
259950,0migration/225346-21fschecks_count12:29:092
259950,0migration/222584-21fschecks_count09:54:102
259950,0migration/219792-21if_eth1.307:39:112
259950,0migration/212274-21aten_r1power_po07:34:082
249950,0watchdog/218672-21ssh09:31:102
249950,0watchdog/218672-21ssh09:31:102
249950,0watchdog/214965-21needreboot09:09:142
19150,0ktimersoftd/10-21swapper/111:54:121
179950,0migration/17854-21latency_hist12:18:551
179950,0migration/14458-21sh07:24:181
179950,0migration/14036-21kthreadcore12:34:131
179950,0migration/132403-21cpuspeed_turbos07:49:071
179950,0migration/125461-21munin-node08:04:181
179950,0migration/116688-21cpuspeed_turbos10:29:091
179950,0migration/115466-21timerwakeupswit09:09:181
179950,0migration/115132-21kthreadcore11:44:131
179950,0migration/110537-21taskset11:20:471
169950,0watchdog/116697-21tr07:34:141
169950,0watchdog/112919-21kthreadcore07:09:101
149950,0migration/09720-21ksmtuned09:25:320
149950,0migration/06602-21aten_r1power_cu10:04:050
149950,0migration/05795-21kthreadcore12:34:140
149950,0migration/05047-21netstat10:19:150
149950,0migration/04653-21kthreadcore10:59:130
149950,0migration/04423-21sh09:43:060
149950,0migration/031479-21sh10:18:420
149950,0migration/030654-21smart_sdb10:34:200
149950,0migration/029464-21kthreadcore11:14:120
149950,0migration/027854-21kernelversion09:19:130
149950,0migration/026700-21cpuspeed_turbos08:09:080
149950,0migration/025839-21if_eth1.212:29:110
149950,0migration/025770-21munin-node09:34:180
149950,0migration/024416-21latency_hist07:18:560
149950,0migration/020842-21kthreadcore09:14:140
149950,0migration/019692-21kthreadcore10:49:130
149950,0migration/018884-21sh09:31:340
149950,0migration/018884-21sh09:31:340
149950,0migration/01862-21sh10:58:380
149950,0migration/017955-21kthreadcore10:29:140
149950,0migration/016877-21kthreadcore08:24:130
149950,0migration/016406-21kthreadcore07:34:140
149950,0migration/016396-21kthreadcore07:09:120
149950,0migration/016387-21taskset10:46:140
149950,0migration/015484-21fschecks_count12:04:100
149950,0migration/012876-21kthreadcore11:04:120
149950,0migration/010436-21kthreadcore09:44:120
109809955,0cyclictest0-21swapper/212:04:202
109809955,0cyclictest0-21swapper/210:13:492
109809950,0cyclictest0-21swapper/210:50:082
109799955,0cyclictest0-21swapper/112:28:251
109799955,0cyclictest0-21swapper/110:21:591
109799955,0cyclictest0-21swapper/109:40:311
109799954,0cyclictest20991-21tr09:14:141
109799950,0cyclictest0-21swapper/112:05:331
109799950,0cyclictest0-21swapper/110:57:461
109799950,0cyclictest0-21swapper/110:04:381
109799950,0cyclictest0-21swapper/110:03:511
109799950,0cyclictest0-21swapper/109:45:431
109799950,0cyclictest0-21swapper/109:27:441
109789950,0cyclictest0-21swapper/011:46:440
109789950,0cyclictest0-21swapper/011:27:580
339940,0migration/38927-21fschecks_count08:44:093
339940,0migration/37006-21kthreadcore09:04:123
339940,0migration/325409-21latency_hist07:43:563
339940,0migration/324981-21taskset07:40:483
339940,0migration/324480-21kthreadcore08:29:193
339940,0migration/319183-21kthreadcore07:14:133
29725040,0irq/27-eth0-tx-31810-21ssh10:18:572
29725040,0irq/27-eth0-tx-25942-21ssh11:10:362
29725040,0irq/27-eth0-tx-23576-21ssh12:09:082
29725040,0irq/27-eth0-tx-2164-21ssh10:39:132
29725040,0irq/27-eth0-tx-0-21swapper/211:39:002
29725040,0irq/27-eth0-tx-0-21swapper/211:14:132
29725040,0irq/27-eth0-tx-0-21swapper/210:43:592
29725040,0irq/27-eth0-tx-0-21swapper/210:34:062
29725040,0irq/27-eth0-tx-0-21swapper/209:39:432
29725040,0irq/27-eth0-tx-0-21swapper/209:34:252
29725040,0irq/27-eth0-tx-0-21swapper/209:14:382
29715040,0irq/26-eth0-rx-0-21swapper/311:09:103
29715040,0irq/26-eth0-rx-0-21swapper/310:20:333
29715040,0irq/26-eth0-rx-0-21swapper/309:59:003
29715040,0irq/26-eth0-rx-0-21swapper/309:49:153
29715040,0irq/26-eth0-rx-0-21swapper/309:45:243
29715040,0irq/26-eth0-rx-0-21swapper/308:39:133
29715040,0irq/26-eth0-rx-0-21swapper/307:59:133
259940,0migration/28693-21kthreadcore11:59:122
259940,0migration/26574-21kthreadcore11:19:122
259940,0migration/232169-21sh07:49:062
259940,0migration/231362-21kthreadcore11:34:132
259940,0migration/230743-21latency07:44:142
259940,0migration/230530-21runrttasks07:19:342
259940,0migration/224307-21kthreadcore08:29:192
259940,0migration/223032-21kthreadcore08:04:132
259940,0migration/21997-21cpuspeed_turbos08:39:082
259940,0migration/218161-21kthreadcore12:24:132
259940,0migration/215699-21kthreadcore08:24:132
259940,0migration/215660-21fschecks_time08:49:092
249940,0watchdog/28668-21kthreadcore07:29:142
249940,0watchdog/23749-21hddtemp_smartct09:24:102
249940,0watchdog/228857-21fschecks_count08:59:102
249940,0watchdog/212918-21kthreadcore07:09:102
249940,0watchdog/20-21swapper/207:54:142
179940,0migration/19865-21fschecks_count10:44:081
179940,0migration/19794-21kthreadcore10:24:111
179940,0migration/16542-21kthreadcore11:39:111
179940,0migration/12939-21kthreadcore08:14:131
179940,0migration/127497-21if_eth109:19:121
179940,0migration/126409-21kthreadcore12:29:131
179940,0migration/123026-21kthreadcore08:29:191
179940,0migration/121820-21kthreadcore07:14:141
179940,0migration/117251-21munin-node09:29:161
179940,0migration/117251-21munin-node09:29:161
179940,0migration/115842-21kthreadcore11:24:131
179940,0migration/114425-21kthreadcore11:04:121
169940,0watchdog/15600-21tr10:59:141
169940,0watchdog/14062-21kthreadcore10:39:141
169940,0watchdog/122958-21ethtool11:49:091
169940,0watchdog/122000-21ssh09:54:021
169940,0watchdog/116523-21kthreadcore09:49:141
169940,0watchdog/10-21swapper/112:21:291
169940,0watchdog/10-21swapper/108:59:101
169940,0watchdog/10-21swapper/108:39:101
169940,0watchdog/10-21swapper/108:09:141
169940,0watchdog/10-21swapper/107:29:141
169940,0watchdog/10-21swapper/107:19:141
149940,0migration/06349-21kthreadcore08:39:130
149940,0migration/032198-21irqcore07:24:120
149940,0migration/032116-21kthreadcore11:34:140
149940,0migration/023772-21kthreadcore11:29:150
149940,0migration/013933-21perf08:23:560
109809943,0cyclictest3514-21snmpd08:21:472
109809943,0cyclictest3514-21snmpd08:13:022
109809943,0cyclictest3514-21snmpd07:25:322
109809943,0cyclictest18818-21awk07:14:122
109809943,0cyclictest14129-21awk07:59:122
109809943,0cyclictest13143-21awk11:24:112
109809940,0cyclictest0-21swapper/211:04:072
109799944,0cyclictest22425-21fschecks_time08:54:121
109799944,0cyclictest0-21swapper/111:31:111
109799943,0cyclictest3577-21awk09:04:091
109799943,0cyclictest3514-21snmpd11:59:031
109799943,0cyclictest3514-21snmpd10:11:031
109799943,0cyclictest24387-21awk10:14:111
109789944,0cyclictest0-21swapper/012:01:330
109789944,0cyclictest0-21swapper/008:58:590
109789943,0cyclictest9425-21awk08:44:110
109789943,0cyclictest7421-21awk07:54:120
109789943,0cyclictest512-21awk07:49:090
109789943,0cyclictest3556-21lldpd07:46:510
109789943,0cyclictest3241-21irqbalance11:39:040
109789943,0cyclictest3241-21irqbalance09:09:040
109789943,0cyclictest31354-21awk09:59:130
109789943,0cyclictest14503-21awk09:49:120
29715030,0irq/26-eth0-rx-0-21swapper/307:51:173
29715030,0irq/26-eth0-rx-0-21swapper/307:24:083
29715030,0irq/26-eth0-rx-0-21swapper/307:11:393
249930,0watchdog/20-21swapper/208:14:142
179930,0migration/127741-21kthreadcore12:09:141
169930,0watchdog/10-21swapper/111:34:051
169930,0watchdog/10-21swapper/110:49:141
169930,0watchdog/10-21swapper/110:34:181
169930,0watchdog/10-21swapper/108:43:581
169930,0watchdog/10-21swapper/107:44:141
109809932,0cyclictest14101-21sensors08:44:172
109809931,0cyclictest23348-21kthreadcore08:54:152
109799933,0cyclictest0-21swapper/109:34:311
109799932,0cyclictest8339-21awk08:19:101
109799932,0cyclictest3556-21lldpd08:01:511
109799932,0cyclictest3514-21snmpd08:34:171
109799932,0cyclictest23037-21cut11:09:131
109799932,0cyclictest17506-21grep08:49:121
109799932,0cyclictest15015-21awk08:24:111
109799930,0cyclictest742-21ssh11:14:151
109789933,0cyclictest22939-21awk11:49:090
109789932,0cyclictest9175-21ps10:24:110
109789932,0cyclictest6597-21sh12:16:180
109789932,0cyclictest3556-21lldpd08:51:510
109789932,0cyclictest3556-21lldpd08:37:510
109789932,0cyclictest30066-21ssh12:10:190
109789932,0cyclictest30042-21sh09:58:400
109789932,0cyclictest24390-21kthreadcore08:29:190
109789932,0cyclictest22921-21kthreadcore08:04:130
109789932,0cyclictest22229-21grep11:09:130
109789932,0cyclictest18676-21awk07:14:110
109789932,0cyclictest1561-21awk08:14:110
109789932,0cyclictest14678-21cut07:59:130
109789930,0cyclictest23006-21ssh10:13:380
29715020,0irq/26-eth0-rx-0-21swapper/307:28:573
169920,0watchdog/10-21swapper/107:38:581
109789920,0cyclictest5363-21wc07:28:560
109789920,0cyclictest0-21swapper/012:23:570
109789920,0cyclictest0-21swapper/009:03:560
109789920,0cyclictest0-21swapper/007:39:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional