You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-24 - 09:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Wed Dec 24, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
625123728,0sleep20-21swapper/219:06:472
625423624,0sleep00-21swapper/019:06:490
620923626,0sleep30-21swapper/319:06:203
623523425,0sleep10-21swapper/119:06:341
65159999,0cyclictest4187-21sensors21:24:202
339970,0migration/35232-21sh23:19:593
339960,0migration/39825-21sh00:21:063
339960,0migration/39708-21munin-node20:44:223
339960,0migration/38420-21sh22:05:403
339960,0migration/330608-21sh21:23:593
339960,0migration/328049-21sh22:55:393
339960,0migration/319716-21taskset19:18:173
339960,0migration/31895-21sh22:22:153
339960,0migration/317652-21taskset00:24:463
29715060,0irq/26-eth0-rx-31691-21taskset23:54:363
29715060,0irq/26-eth0-rx-23287-21tune2fs23:14:113
29715060,0irq/26-eth0-rx-18558-21port_sshpriv22:49:183
259960,0migration/212485-21irqcore22:29:152
149960,0migration/030449-21sh23:17:580
149960,0migration/019139-21gltestperf00:09:120
8215050,0irq/24-0000:00:4536-21munin-run21:08:592
65159955,0cyclictest0-21swapper/223:06:192
65159955,0cyclictest0-21swapper/222:37:282
65159955,0cyclictest0-21swapper/221:50:482
65159955,0cyclictest0-21swapper/221:19:252
65159950,0cyclictest0-21swapper/219:27:292
65139953,0cyclictest8060-21kworker/0:000:30:190
65139953,0cyclictest8060-21kworker/0:000:19:010
65139953,0cyclictest29408-21kworker/0:223:10:100
65139953,0cyclictest29408-21kworker/0:223:04:100
65139953,0cyclictest29408-21kworker/0:223:03:020
65139953,0cyclictest29408-21kworker/0:222:51:500
65139953,0cyclictest29408-21kworker/0:222:45:320
65139953,0cyclictest29408-21kworker/0:222:30:500
65139953,0cyclictest29408-21kworker/0:222:19:320
65139953,0cyclictest29408-21kworker/0:222:15:260
65139953,0cyclictest29408-21kworker/0:222:09:400
65139953,0cyclictest29408-21kworker/0:222:04:060
65139953,0cyclictest29408-21kworker/0:221:59:120
65139953,0cyclictest29408-21kworker/0:221:45:560
65139953,0cyclictest29408-21kworker/0:221:41:560
65139953,0cyclictest29408-21kworker/0:221:29:500
65139953,0cyclictest29408-21kworker/0:221:14:000
65139952,0cyclictest8060-21kworker/0:023:39:390
65139952,0cyclictest8060-21kworker/0:023:28:490
65139952,0cyclictest29408-21kworker/0:222:54:320
65139952,0cyclictest29408-21kworker/0:221:52:420
35150,0ktimersoftd/30-21swapper/321:51:483
339950,0migration/37602-21munin-node19:54:203
339950,0migration/36302-21munin-node23:04:113
339950,0migration/332376-21switchtime19:24:253
339950,0migration/329055-21munin-node00:34:113
339950,0migration/32812-21kthreadcore21:44:153
339950,0migration/327065-21kthreadcore21:59:153
339950,0migration/32698-21latency_hist20:18:593
339950,0migration/326304-21kthreadcore21:19:163
339950,0migration/322186-21kthreadcore23:49:153
339950,0migration/317214-21aten_r1power_en20:54:073
339950,0migration/316815-21kthreadcore19:39:143
339950,0migration/31254-21df_abs19:29:123
339950,0migration/312486-21irqcore22:29:153
339950,0migration/311365-21munin-node20:49:143
339950,0migration/311230-21taskset21:10:263
339950,0migration/310661-21kthreadcore19:59:173
29725050,0irq/27-eth0-tx-782-21tune2fs21:44:112
29725050,0irq/27-eth0-tx-7270-21munin-node23:24:122
29725050,0irq/27-eth0-tx-6250-21ssh23:22:212
29725050,0irq/27-eth0-tx-4544-21diskmemload21:33:432
29725050,0irq/27-eth0-tx-364-21ssh22:01:582
29725050,0irq/27-eth0-tx-28597-21munin-node20:14:150
29725050,0irq/27-eth0-tx-26915-21munin-node22:19:182
29725050,0irq/27-eth0-tx-26789-21munin-node00:14:202
29725050,0irq/27-eth0-tx-2665-21latency_hist22:23:592
29725050,0irq/27-eth0-tx-23837-21munin-node20:59:210
29725050,0irq/27-eth0-tx-16913-21munin-node00:24:192
29725050,0irq/27-eth0-tx-10139-21munin-node22:09:142
29725050,0irq/27-eth0-tx-0-21swapper/223:54:392
29725050,0irq/27-eth0-tx-0-21swapper/223:42:002
29725050,0irq/27-eth0-tx-0-21swapper/223:31:282
29725050,0irq/27-eth0-tx-0-21swapper/223:14:442
29725050,0irq/27-eth0-tx-0-21swapper/222:40:112
29725050,0irq/27-eth0-tx-0-21swapper/222:40:112
29725050,0irq/27-eth0-tx-0-21swapper/200:08:502
29725050,0irq/27-eth0-tx-0-21swapper/200:03:332
29725050,0irq/27-eth0-tx-0-21swapper/021:09:160
29725050,0irq/27-eth0-tx-0-21swapper/020:43:340
29725050,0irq/27-eth0-tx-0-21swapper/019:29:110
29725050,0irq/27-eth0-tx-0-21swapper/019:18:190
29715050,0irq/26-eth0-rx-5951-21sshd23:39:213
29715050,0irq/26-eth0-rx-30833-21ssh22:59:173
29715050,0irq/26-eth0-rx-2590-21ssh22:40:343
29715050,0irq/26-eth0-rx-2590-21ssh22:40:343
29715050,0irq/26-eth0-rx-19393-21cut23:29:163
29715050,0irq/26-eth0-rx-17658-21tr21:34:153
29715050,0irq/26-eth0-rx-12070-21kthreadcore00:04:173
29715050,0irq/26-eth0-rx-11027-21kthreadcore19:09:173
29715050,0irq/26-eth0-rx-0-21swapper/323:33:583
29715050,0irq/26-eth0-rx-0-21swapper/323:11:453
29715050,0irq/26-eth0-rx-0-21swapper/322:44:163
29715050,0irq/26-eth0-rx-0-21swapper/322:10:163
29715050,0irq/26-eth0-rx-0-21swapper/321:56:123
29715050,0irq/26-eth0-rx-0-21swapper/321:42:013
29715050,0irq/26-eth0-rx-0-21swapper/300:14:203
29715050,0irq/26-eth0-rx-0-21swapper/300:09:153
259950,0migration/25324-21kthreadcore19:54:162
259950,0migration/230289-21sh23:34:272
259950,0migration/22720-21latency_hist20:18:592
259950,0migration/224670-21sh22:14:232
259950,0migration/218599-21kthreadcore21:54:152
259950,0migration/218317-21sh21:34:162
259950,0migration/213020-21kthreadcore23:44:172
169950,0watchdog/116025-21kthreadcore22:29:171
169950,0watchdog/10-21swapper/122:34:171
169950,0watchdog/10-21swapper/119:44:211
149950,0migration/09998-21kthreadcore19:34:150
149950,0migration/08584-21ntp_kernel_pll_22:24:180
149950,0migration/07337-21fschecks_time19:09:130
149950,0migration/03496-21kthreadcore23:19:160
149950,0migration/031353-21taskset23:36:580
149950,0migration/030932-21cpuspeed_turbos21:24:110
149950,0migration/030459-21kthreadcore00:34:130
149950,0migration/029044-21kthreadcore23:54:170
149950,0migration/02621-21irqcore23:59:160
149950,0migration/025988-21kthreadcore20:34:170
149950,0migration/024330-21kthreadcore21:19:150
149950,0migration/023392-21kthreadcore19:44:210
149950,0migration/022810-21munin-node23:49:190
149950,0migration/019457-21kthreadcore21:34:160
149950,0migration/017725-21taskset00:08:100
149950,0migration/017088-21kthreadcore20:04:150
149950,0migration/01626-21kthreadcore21:04:160
149950,0migration/01343-21taskset19:50:370
65159943,0cyclictest3556-21lldpd19:21:472
65159943,0cyclictest3514-21snmpd20:59:272
65159943,0cyclictest3514-21snmpd19:35:572
65159943,0cyclictest3514-21snmpd19:29:122
65159943,0cyclictest21864-21awk19:44:192
65149943,0cyclictest9576-21awk21:49:141
65149943,0cyclictest9499-21awk19:59:151
65149943,0cyclictest3556-21lldpd23:56:471
65149943,0cyclictest3556-21lldpd22:49:171
65149943,0cyclictest3556-21lldpd22:08:461
65149943,0cyclictest3556-21lldpd21:55:161
65149943,0cyclictest3556-21lldpd19:55:471
65149943,0cyclictest3556-21lldpd19:39:471
65149943,0cyclictest3514-21snmpd21:13:421
65149943,0cyclictest3514-21snmpd20:40:121
65149943,0cyclictest3514-21snmpd20:18:571
65149943,0cyclictest3514-21snmpd00:17:291
65149943,0cyclictest3241-21irqbalance23:59:461
65149943,0cyclictest3241-21irqbalance22:39:071
65149943,0cyclictest3241-21irqbalance22:39:071
65149943,0cyclictest3241-21irqbalance22:24:071
65149943,0cyclictest3241-21irqbalance22:19:171
65149943,0cyclictest3241-21irqbalance22:14:471
65149943,0cyclictest3241-21irqbalance21:24:071
65149943,0cyclictest3241-21irqbalance20:24:471
65149943,0cyclictest3241-21irqbalance20:20:171
65149943,0cyclictest3241-21irqbalance19:34:171
65149943,0cyclictest31313-21awk21:04:131
65149943,0cyclictest28465-21awk19:49:151
65149943,0cyclictest27651-21awk19:24:161
65149943,0cyclictest26509-21awk21:59:141
65149943,0cyclictest23807-21awk20:34:151
65149943,0cyclictest20887-21awk19:19:131
65149943,0cyclictest20597-21awk00:29:121
65149943,0cyclictest19301-21awk00:09:131
65149943,0cyclictest17208-21awk20:29:151
65149943,0cyclictest16503-21awk21:34:141
65149943,0cyclictest14143-21awk21:14:131
65149943,0cyclictest11093-21awk22:09:161
65149943,0cyclictest1055-21awk21:44:131
65149940,0cyclictest17576-21diskstats20:54:091
65139943,0cyclictest3556-21lldpd22:39:470
65139943,0cyclictest3556-21lldpd22:39:470
65139943,0cyclictest18135-21awk20:54:120
65139943,0cyclictest12350-21awk00:24:140
65139942,0cyclictest8060-21kworker/0:023:44:130
65139942,0cyclictest8060-21kworker/0:000:17:550
65139942,0cyclictest29408-21kworker/0:222:34:560
65139942,0cyclictest29408-21kworker/0:221:54:020
339940,0migration/39177-21taskset20:22:103
339940,0migration/38047-21if_eth1.321:29:153
339940,0migration/326723-21switchtime19:44:263
339940,0migration/318305-21kthreadcore20:04:163
339940,0migration/310662-21kthreadcore19:34:153
29725040,0irq/27-eth0-tx-29713-21ssh22:59:002
29725040,0irq/27-eth0-tx-20788-21ssh23:10:042
29725040,0irq/27-eth0-tx-0-21swapper/222:54:102
29725040,0irq/27-eth0-tx-0-21swapper/222:49:042
29725040,0irq/27-eth0-tx-0-21swapper/222:04:082
29725040,0irq/27-eth0-tx-0-21swapper/221:39:142
29725040,0irq/27-eth0-tx-0-21swapper/221:14:072
29725040,0irq/27-eth0-tx-0-21swapper/200:29:022
29725040,0irq/27-eth0-tx-0-21swapper/200:19:092
29725040,0irq/27-eth0-tx-0-21swapper/200:09:122
29725040,0irq/27-eth0-tx-0-21swapper/020:24:120
29725040,0irq/27-eth0-tx-0-21swapper/020:19:170
29725040,0irq/27-eth0-tx-0-21swapper/019:24:130
29725040,0irq/27-eth0-tx-0-21swapper/019:19:200
29715040,0irq/26-eth0-rx-30227-21grep20:39:153
29715040,0irq/26-eth0-rx-27961-21tr20:59:203
29715040,0irq/26-eth0-rx-0-21swapper/323:59:173
29715040,0irq/26-eth0-rx-0-21swapper/323:44:303
29715040,0irq/26-eth0-rx-0-21swapper/323:25:413
29715040,0irq/26-eth0-rx-0-21swapper/322:34:283
29715040,0irq/26-eth0-rx-0-21swapper/322:24:163
29715040,0irq/26-eth0-rx-0-21swapper/322:13:593
29715040,0irq/26-eth0-rx-0-21swapper/321:14:333
29715040,0irq/26-eth0-rx-0-21swapper/320:09:253
29715040,0irq/26-eth0-rx-0-21swapper/319:23:593
29715040,0irq/26-eth0-rx-0-21swapper/300:32:593
259940,0migration/29648-21kthreadcore22:44:172
259940,0migration/29598-21switchtime20:44:212
259940,0migration/227988-21kthreadcore20:34:172
259940,0migration/222349-21latency23:49:152
259940,0migration/215730-21kthreadcore19:39:142
259940,0migration/213607-21kthreadcore20:49:172
259940,0migration/210031-21fschecks_count20:24:132
259940,0migration/210009-21kthreadcore19:59:172
249940,0watchdog/229181-21iostat_ios00:34:122
249940,0watchdog/228897-21irqstats19:49:172
249940,0watchdog/227373-21munin-node20:09:212
249940,0watchdog/219645-21cut20:29:172
249940,0watchdog/218932-21cut20:54:132
249940,0watchdog/214696-21kthreadcore19:14:172
249940,0watchdog/20-21swapper/221:09:172
179940,0migration/15837-21munin-node23:39:201
179940,0migration/118148-21kthreadcore20:04:161
179940,0migration/116523-21idleruntime23:29:131
169940,0watchdog/19916-21irqstats23:44:161
169940,0watchdog/19170-21grep21:29:171
169940,0watchdog/18701-21grep22:44:161
169940,0watchdog/172482sleep123:24:041
169940,0watchdog/16678-21python19:29:211
169940,0watchdog/129196-21kthreadcore23:34:161
169940,0watchdog/128668-21ssh21:20:251
169940,0watchdog/110320-21ssh00:22:161
169940,0watchdog/10-21swapper/123:19:001
169940,0watchdog/10-21swapper/123:09:161
169940,0watchdog/10-21swapper/121:39:331
169940,0watchdog/10-21swapper/100:34:161
149940,0migration/07937-21kthreadcore20:44:150
149940,0migration/024112-21kthreadcore20:09:190
149940,0migration/017245-21kthreadcore23:29:150
149940,0migration/016597-21kthreadcore19:39:140
65159932,0cyclictest3863-21awk20:19:162
65159932,0cyclictest21713-21sadc20:08:592
65149932,0cyclictest4965-21cut20:44:141
65149932,0cyclictest3556-21lldpd20:59:171
65149932,0cyclictest3514-21snmpd20:09:121
65149930,0cyclictest0-21swapper/123:04:061
65149930,0cyclictest0-21swapper/100:24:021
65139933,0cyclictest29408-21kworker/0:220:49:000
65139933,0cyclictest29408-21kworker/0:220:29:060
65139933,0cyclictest29408-21kworker/0:219:59:000
65139933,0cyclictest29408-21kworker/0:219:54:000
29715030,0irq/26-eth0-rx-0-21swapper/320:34:123
29715030,0irq/26-eth0-rx-0-21swapper/320:29:113
29715030,0irq/26-eth0-rx-0-21swapper/320:24:113
29715030,0irq/26-eth0-rx-0-21swapper/319:49:123
249930,0watchdog/230539-21ps20:39:172
249930,0watchdog/20-21swapper/219:09:172
169930,0watchdog/14544-21diskmemload23:14:081
169930,0watchdog/114473-21cut20:49:171
169930,0watchdog/10-21swapper/123:50:001
169930,0watchdog/10-21swapper/123:00:441
169930,0watchdog/10-21swapper/122:54:121
169930,0watchdog/10-21swapper/119:14:171
169930,0watchdog/10-21swapper/119:09:131
169930,0watchdog/10-21swapper/100:07:081
29715020,0irq/26-eth0-rx-0-21swapper/321:03:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional