You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 16:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Tue Apr 14, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26670632990,2sleep20-21swapper/209:40:232
139099960,58rtkit-daemon0-21swapper/307:06:503
261096828148,22sleep10-21swapper/107:05:121
261157326544,14sleep20-21swapper/207:05:202
261228626343,13sleep00-21swapper/007:08:040
26324222490,4sleep20-21swapper/208:05:122
26719892380,4sleep33056-21PK-Backend09:55:163
26895182360,2sleep03056-21PK-Backend10:45:130
26418342270,9sleep126-21ksoftirqd/108:30:201
2612566992614,10cyclictest2663848-21cron09:35:000
261257099254,8cyclictest2699350-21perf-exec11:14:591
2612570992512,3cyclictest2175-21JS10:27:481
139099250,4rtkit-daemon1389-21rtkit-daemon11:36:400
2612574992417,5cyclictest883-21dbus-daemon09:38:032
261257099244,7cyclictest2665608-21idleruntime-cro09:40:001
26220022230,2sleep33056-21PK-Backend07:35:143
2612574992321,1cyclictest0-21swapper/208:48:312
261257499230,3cyclictest2657558-21df_inode09:15:132
2612570992315,6cyclictest2676493-21cron10:10:001
139099230,15rtkit-daemon0-21swapper/207:30:422
2612577992213,7cyclictest2690652-21cron10:50:013
2612577992211,9cyclictest42-21ksoftirqd/312:35:003
2612574992220,1cyclictest0-21swapper/211:08:362
2612574992220,1cyclictest0-21swapper/211:03:292
2612574992220,1cyclictest0-21swapper/209:33:332
2612574992220,1cyclictest0-21swapper/208:04:302
2612574992220,1cyclictest0-21swapper/207:28:472
261257499220,20cyclictest0-21swapper/211:11:412
261257099222,7cyclictest26-21ksoftirqd/107:20:101
2612570992220,1cyclictest0-21swapper/112:10:521
2612570992220,1cyclictest0-21swapper/109:08:361
2612570992220,1cyclictest0-21swapper/108:03:251
2612570992220,1cyclictest0-21swapper/107:11:421
2612570992220,1cyclictest0-21swapper/107:11:411
2612570992219,2cyclictest3056-21PK-Backend11:45:031
2612570992219,2cyclictest0-21swapper/112:31:351
2612566992220,1cyclictest0-21swapper/012:38:040
2612566992220,1cyclictest0-21swapper/010:00:340
2612566992220,1cyclictest0-21swapper/008:58:570
2612566992220,1cyclictest0-21swapper/008:00:500
2612566992218,2cyclictest2624765-21sed07:40:210
261256699220,20cyclictest2682879-21expr10:25:210
261257799216,13cyclictest2663889-21systemd-run09:35:013
2612577992117,2cyclictest42-21ksoftirqd/308:44:593
2612574992119,1cyclictest0-21swapper/211:43:452
2612574992119,1cyclictest0-21swapper/211:30:532
2612574992119,1cyclictest0-21swapper/210:50:252
2612574992119,1cyclictest0-21swapper/209:20:272
2612574992119,1cyclictest0-21swapper/209:11:022
2612574992119,1cyclictest0-21swapper/208:28:572
2612574992119,1cyclictest0-21swapper/207:21:242
2612574992118,2cyclictest3056-21PK-Backend08:10:182
2612574992116,3cyclictest2726476-21python312:30:212
2612574992113,6cyclictest2710703-21ntp_kernel_err11:45:182
2612574992112,7cyclictest2667979-21gpgconf09:45:012
261257499210,20cyclictest0-21swapper/212:27:312
261257499210,20cyclictest0-21swapper/210:56:202
261257499210,20cyclictest0-21swapper/210:48:222
261257499210,20cyclictest0-21swapper/210:36:322
261257499210,20cyclictest0-21swapper/210:30:382
261257499210,20cyclictest0-21swapper/210:07:122
261257499210,20cyclictest0-21swapper/209:59:282
261257499210,20cyclictest0-21swapper/209:02:232
261257499210,20cyclictest0-21swapper/207:36:392
261257499210,20cyclictest0-21swapper/207:16:382
261257499210,20cyclictest0-21swapper/207:14:142
261257499210,20cyclictest0-21swapper/207:14:142
261257499210,19cyclictest0-21swapper/208:50:192
261257099215,5cyclictest3056-21PK-Backend08:20:091
2612570992119,1cyclictest0-21swapper/112:03:091
2612570992119,1cyclictest0-21swapper/111:32:271
2612570992119,1cyclictest0-21swapper/111:00:371
2612570992119,1cyclictest0-21swapper/109:40:331
2612570992119,1cyclictest0-21swapper/108:47:201
2612570992119,1cyclictest0-21swapper/107:51:571
2612570992119,1cyclictest0-21swapper/107:37:381
2612570992117,2cyclictest2619722-21taskset07:27:241
261257099210,20cyclictest3056-21PK-Backend09:34:171
261257099210,20cyclictest0-21swapper/112:35:121
261257099210,20cyclictest0-21swapper/111:08:341
261257099210,20cyclictest0-21swapper/109:56:071
261257099210,20cyclictest0-21swapper/108:37:401
261257099210,1cyclictest3056-21PK-Backend09:15:251
2612566992119,1cyclictest0-21swapper/008:09:070
2612566992112,7cyclictest883-21dbus-daemon10:15:330
2612566992112,7cyclictest883-21dbus-daemon10:15:330
261256699210,20cyclictest0-21swapper/012:27:020
261256699210,20cyclictest0-21swapper/012:21:480
261256699210,20cyclictest0-21swapper/009:59:330
2612577992018,1cyclictest0-21swapper/310:40:183
2612577992015,4cyclictest42-21ksoftirqd/311:50:013
2612577992015,3cyclictest42-21ksoftirqd/311:45:013
2612577992013,5cyclictest2664263-21expr09:35:143
2612577992011,7cyclictest2692389-21cron10:55:003
2612577992011,3cyclictest42-21ksoftirqd/309:45:003
261257799200,19cyclictest3056-21PK-Backend08:24:453
2612574992018,1cyclictest3056-21PK-Backend10:20:372
2612574992018,1cyclictest0-21swapper/212:08:112
2612574992018,1cyclictest0-21swapper/211:58:242
2612574992018,1cyclictest0-21swapper/211:53:322
2612574992018,1cyclictest0-21swapper/211:15:462
2612574992018,1cyclictest0-21swapper/209:08:252
2612574992018,1cyclictest0-21swapper/208:42:152
2612574992018,1cyclictest0-21swapper/208:30:462
2612574992011,7cyclictest2637605-21apt08:20:122
261257499200,1cyclictest0-21swapper/212:37:382
261257499200,19cyclictest0-21swapper/212:21:512
261257499200,19cyclictest0-21swapper/212:16:572
261257499200,19cyclictest0-21swapper/211:27:492
261257499200,19cyclictest0-21swapper/210:42:262
261257499200,19cyclictest0-21swapper/210:27:342
261257499200,19cyclictest0-21swapper/210:11:032
261257499200,19cyclictest0-21swapper/209:26:292
261257499200,19cyclictest0-21swapper/208:36:002
261257499200,19cyclictest0-21swapper/207:51:592
261257099203,6cyclictest26-21ksoftirqd/111:22:501
2612570992018,1cyclictest3056-21PK-Backend11:57:281
2612570992018,1cyclictest3056-21PK-Backend11:44:021
2612570992018,1cyclictest3056-21PK-Backend11:37:191
2612570992018,1cyclictest0-21swapper/112:21:281
2612570992018,1cyclictest0-21swapper/110:58:341
2612570992018,1cyclictest0-21swapper/109:04:231
2612570992018,1cyclictest0-21swapper/107:30:471
2612570992013,5cyclictest2704030-21cat11:25:231
2612570992013,5cyclictest2624100-21df07:40:131
261257099201,10cyclictest2635432-21runrttasks08:13:081
2612570992010,7cyclictest436-20systemd-journal08:29:591
261257099200,1cyclictest0-21swapper/108:42:541
261257099200,19cyclictest3056-21PK-Backend10:40:011
261257099200,19cyclictest3056-21PK-Backend10:31:161
261257099200,19cyclictest3056-21PK-Backend09:26:261
261257099200,19cyclictest0-21swapper/112:19:241
261257099200,19cyclictest0-21swapper/111:17:321
261257099200,19cyclictest0-21swapper/107:46:301
2612566992018,1cyclictest3056-21PK-Backend10:50:080
2612566992018,1cyclictest0-21swapper/012:04:520
2612566992018,1cyclictest0-21swapper/009:22:490
2612566992018,1cyclictest0-21swapper/008:48:080
2612566992018,1cyclictest0-21swapper/008:32:340
261256699200,19cyclictest0-21swapper/011:16:040
261256699200,19cyclictest0-21swapper/010:12:520
261256699200,19cyclictest0-21swapper/008:42:190
261256699200,19cyclictest0-21swapper/007:13:450
261256699200,19cyclictest0-21swapper/007:13:450
139099200,3rtkit-daemon1389-21rtkit-daemon09:53:500
139099200,14rtkit-daemon3056-21PK-Backend07:28:433
261257799195,3cyclictest42-21ksoftirqd/309:45:233
2612577991917,1cyclictest3056-21PK-Backend07:43:003
2612577991914,3cyclictest2636726-21python308:15:223
2612577991912,6cyclictest2680248-21awk10:20:103
2612577991911,6cyclictest2659290-21df09:20:143
2612574991917,1cyclictest0-21swapper/212:03:172
2612574991917,1cyclictest0-21swapper/211:37:492
261257499190,1cyclictest3056-21PK-Backend10:00:302
261257499190,1cyclictest0-21swapper/212:12:042
261257499190,1cyclictest0-21swapper/209:52:462
261257499190,1cyclictest0-21swapper/207:56:392
261257099192,6cyclictest26-21ksoftirqd/110:25:491
2612570991917,1cyclictest3056-21PK-Backend11:51:471
2612570991917,1cyclictest3056-21PK-Backend09:22:151
2612570991917,1cyclictest0-21swapper/112:07:001
2612570991917,1cyclictest0-21swapper/108:59:231
2612570991917,1cyclictest0-21swapper/108:51:461
2612570991917,1cyclictest0-21swapper/107:17:031
2612570991914,3cyclictest2677441-21tail10:10:181
261257099190,6cyclictest3056-21PK-Backend09:45:071
261257099190,1cyclictest0-21swapper/112:26:081
261257099190,1cyclictest0-21swapper/110:52:381
261257099190,1cyclictest0-21swapper/110:15:471
261257099190,1cyclictest0-21swapper/110:15:471
261257099190,1cyclictest0-21swapper/107:55:361
261257099190,17cyclictest2655667-21cstates09:10:131
2612566991917,1cyclictest0-21swapper/012:34:400
2612566991917,1cyclictest0-21swapper/012:07:290
2612566991912,6cyclictest2714332-21python311:55:200
2612566991912,5cyclictest2617190-21cat07:20:160
261256699190,1cyclictest3056-21PK-Backend10:43:060
261256699190,1cyclictest0-21swapper/011:11:250
261256699190,1cyclictest0-21swapper/010:08:250
261256699190,1cyclictest0-21swapper/007:45:220
261256699190,1cyclictest0-21swapper/007:36:570
261256699190,18cyclictest0-21swapper/007:59:010
139099190,13rtkit-daemon0-21swapper/207:45:342
139099190,13rtkit-daemon0-21swapper/207:43:262
26377002180,3chrt0-21swapper/008:20:140
2612577991814,3cyclictest42-21ksoftirqd/311:55:003
2612577991814,3cyclictest42-21ksoftirqd/309:14:593
2612577991813,3cyclictest2662155-21munin-run09:30:013
2612577991813,3cyclictest2658686-21munin-run09:19:593
2612577991811,5cyclictest978-21snmpd10:18:393
2612577991811,5cyclictest978-21snmpd10:18:393
2612577991811,5cyclictest2613224-21cat07:10:153
2612577991811,5cyclictest2613224-21cat07:10:153
2612577991810,6cyclictest42-21ksoftirqd/311:40:013
2612577991810,6cyclictest2714240-21awk11:55:183
2612577991810,2cyclictest42-21ksoftirqd/312:00:123
261257799180,6cyclictest2683992-21apt10:30:123
2612574991816,1cyclictest0-21swapper/208:15:172
2612570991816,1cyclictest3056-21PK-Backend09:51:271
2612570991816,1cyclictest0-21swapper/110:03:381
261257099180,6cyclictest2681463-21runrttasks10:21:551
261257099180,6cyclictest26-21ksoftirqd/110:45:451
261257099180,6cyclictest26-21ksoftirqd/110:35:181
2612566991816,1cyclictest0-21swapper/011:28:010
2612566991814,3cyclictest13-21ksoftirqd/008:30:010
2612566991812,5cyclictest13-21ksoftirqd/011:00:120
2612566991812,4cyclictest2681404-21sshd10:21:070
261256699180,1cyclictest0-21swapper/007:27:580
261257799176,9cyclictest42-21ksoftirqd/311:13:563
261257799174,2cyclictest141rcu_preempt07:30:093
261257799172,6cyclictest2720128-21sh12:15:003
2612577991712,4cyclictest978-21snmpd10:28:143
2612577991712,4cyclictest42-21ksoftirqd/310:40:013
2612577991712,3cyclictest42-21ksoftirqd/310:12:243
2612577991711,4cyclictest2673808-21sed10:00:173
261257799171,14cyclictest2670584-21ntp_states09:50:193
2612574991715,1cyclictest0-21swapper/210:17:472
2612574991715,1cyclictest0-21swapper/210:17:462
2612574991715,1cyclictest0-21swapper/208:58:102
261257099170,6cyclictest2636961-21taskset08:15:291
261256699177,2cyclictest141rcu_preempt09:45:230
2612566991715,1cyclictest0-21swapper/011:20:440
2612566991715,1cyclictest0-21swapper/009:15:050
2612566991715,1cyclictest0-21swapper/008:54:100
2612566991715,1cyclictest0-21swapper/007:18:510
2612566991712,4cyclictest2711873-21tr11:50:120
2612566991712,4cyclictest13-21ksoftirqd/007:50:220
2612566991711,5cyclictest1968-21Xorg09:00:560
2612566991711,4cyclictest2709720-21cron11:45:010
2612566991711,4cyclictest2684127-21cpuspeed_turbos10:30:130
2612566991710,6cyclictest2718852-21tr12:10:120
2612566991710,6cyclictest2662111-21cron09:30:000
2612566991710,5cyclictest2696633-21cut11:05:160
2612566991710,5cyclictest2686453-21cat10:35:190
2612566991710,5cyclictest2655864-21ls09:10:140
2612566991710,5cyclictest2642788-21cat08:35:110
27108122160,4sleep00-21swapper/011:45:190
26541852160,2sleep00-21swapper/009:05:160
261257799169,5cyclictest42-21ksoftirqd/307:50:193
2612577991612,2cyclictest42-21ksoftirqd/310:08:223
261257799161,13cyclictest2716911-21/usr/sbin/munin12:05:213
261257799160,8cyclictest3056-21PK-Backend07:50:003
261257799160,14cyclictest3056-21PK-Backend11:30:543
261257799160,14cyclictest3056-21PK-Backend08:00:043
261257799160,14cyclictest2633277-21switchtime08:05:203
2612574991612,3cyclictest2701310-21/usr/sbin/munin11:20:182
261256699169,6cyclictest2705786-21sed11:30:210
2612566991614,1cyclictest0-21swapper/008:16:520
261256699160,15cyclictest0-21swapper/012:19:590
261257799159,4cyclictest42-21ksoftirqd/308:30:213
261257799154,2cyclictest141rcu_preempt07:20:083
2612577991513,1cyclictest3056-21PK-Backend07:15:083
2612577991511,3cyclictest3056-21PK-Backend08:36:413
2612577991511,3cyclictest3056-21PK-Backend08:10:573
2612577991511,2cyclictest42-21ksoftirqd/311:27:463
2612577991511,2cyclictest42-21ksoftirqd/311:18:513
2612577991511,2cyclictest42-21ksoftirqd/311:03:473
2612577991511,2cyclictest42-21ksoftirqd/309:00:233
2612577991511,2cyclictest42-21ksoftirqd/308:50:363
2612577991510,3cyclictest26939802sleep310:58:193
261257799150,6cyclictest3056-21PK-Backend12:18:243
261257799150,5cyclictest0-21swapper/308:25:203
261256699159,4cyclictest2634743-21missed_timers08:10:180
261256699156,7cyclictest883-21dbus-daemon07:34:350
2612566991512,2cyclictest2667575-21sh09:44:590
2612566991510,4cyclictest13-21ksoftirqd/010:55:200
261257799145,8cyclictest42-21ksoftirqd/307:55:143
261257799145,2cyclictest141rcu_preempt12:25:093
261257799142,2cyclictest141rcu_preempt12:20:223
2612577991410,3cyclictest3056-21PK-Backend09:05:283
2612577991410,3cyclictest3056-21PK-Backend08:45:063
2612577991410,2cyclictest42-21ksoftirqd/312:35:283
2612577991410,2cyclictest42-21ksoftirqd/311:05:383
2612577991410,2cyclictest42-21ksoftirqd/308:56:013
261257799140,3cyclictest141rcu_preempt11:20:243
2612566991310,2cyclictest2665008-21switchtime09:35:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional