You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 19:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Jun 12, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
246526822570,9sleep10-21swapper/107:05:271
257985621220,5sleep30-21swapper/312:35:003
246683821120,4sleep20-21swapper/207:10:202
256139721100,1sleep00-21swapper/011:40:160
24760012990,2sleep20-21swapper/207:35:222
246559127040,24sleep30-21swapper/307:09:003
246549026645,14sleep00-21swapper/007:07:400
246559826545,13sleep20-21swapper/207:09:042
25453852520,3sleep126-21ksoftirqd/110:55:131
25800972480,3sleep10-21swapper/112:35:101
25755452420,4sleep23056-21PK-Backend12:20:202
25160982400,3sleep10-21swapper/109:30:161
24999552390,2sleep03056-21PK-Backend08:45:120
25485212370,2sleep10-21swapper/111:05:081
24715692340,2sleep00-21swapper/007:25:010
2465812993326,5cyclictest2503586-21df08:55:122
2465807992916,10cyclictest2480198-21unin-run07:50:001
246581499270,26cyclictest0-21swapper/308:55:123
139099270,4rtkit-daemon1389-21rtkit-daemon11:50:001
2465812992613,4cyclictest34-21ksoftirqd/212:20:002
2465804992622,3cyclictest2579542-21nm-dispatcher12:32:270
139099260,4rtkit-daemon1389-21rtkit-daemon09:16:040
139099250,4rtkit-daemon1389-21rtkit-daemon10:34:210
139099250,4rtkit-daemon1389-21rtkit-daemon09:44:071
246580799241,11cyclictest24945272apt-get08:30:031
246580799240,5cyclictest2532925-21/usr/sbin/munin10:20:191
2465804992416,6cyclictest2569331-21dump-pmu-power12:05:000
2465812992317,4cyclictest1895067-21snapd07:56:132
2465807992321,1cyclictest0-21swapper/111:34:221
2465807992316,5cyclictest2476692-21cron07:40:001
2465807992312,8cyclictest436-20systemd-journal10:00:001
2465807992312,8cyclictest436-20systemd-journal10:00:001
25295142220,3sleep10-21swapper/110:10:111
25016402220,5sleep3411rcuc/308:50:123
24873862220,3chrt0-21swapper/208:10:152
2465814992218,2cyclictest3056-21PK-Backend11:02:463
246581499220,20cyclictest42-21ksoftirqd/307:58:543
2465812992213,7cyclictest2468141-21fw_packets07:15:022
246580799222,2cyclictest141rcu_preempt07:55:001
246580799222,15cyclictest2493431-21chrt08:25:201
2465807992220,1cyclictest0-21swapper/110:42:451
2465807992220,1cyclictest0-21swapper/109:19:101
246580799221,12cyclictest26-21ksoftirqd/112:25:111
246580799220,20cyclictest3056-21PK-Backend08:52:331
2465804992220,1cyclictest0-21swapper/011:50:490
2465804992220,1cyclictest0-21swapper/011:19:330
2465804992220,1cyclictest0-21swapper/010:16:060
2465804992220,1cyclictest0-21swapper/008:37:370
2465804992219,2cyclictest0-21swapper/008:58:470
2465804992219,2cyclictest0-21swapper/008:52:310
25511412214,2sleep02551337-21sed11:10:220
2465814992115,4cyclictest2500625-21python308:45:223
246581499210,20cyclictest3056-21PK-Backend10:30:493
246581499210,19cyclictest3056-21PK-Backend11:58:223
246581499210,19cyclictest3056-21PK-Backend11:39:023
246581499210,19cyclictest3056-21PK-Backend11:26:043
246581499210,19cyclictest3056-21PK-Backend10:43:463
246581499210,19cyclictest3056-21PK-Backend10:02:373
246581499210,19cyclictest3056-21PK-Backend09:53:373
246581499210,19cyclictest3056-21PK-Backend09:45:213
246581499210,19cyclictest3056-21PK-Backend09:32:233
246581499210,19cyclictest3056-21PK-Backend08:01:453
2465812992119,1cyclictest0-21swapper/210:11:162
246581299210,9cyclictest25801012cat12:35:102
246581299210,20cyclictest0-21swapper/208:21:392
246581299210,20cyclictest0-21swapper/207:28:192
2465807992119,1cyclictest0-21swapper/112:22:341
2465807992119,1cyclictest0-21swapper/112:04:301
2465807992119,1cyclictest0-21swapper/111:43:411
2465807992119,1cyclictest0-21swapper/110:33:511
2465807992118,2cyclictest3056-21PK-Backend09:01:111
2465807992116,4cyclictest26-21ksoftirqd/111:20:001
2465807992114,5cyclictest2507355-21grep09:05:171
246580799210,20cyclictest3056-21PK-Backend12:33:101
246580799210,20cyclictest0-21swapper/111:02:591
246580799210,20cyclictest0-21swapper/109:10:031
246580799210,19cyclictest3056-21PK-Backend07:19:181
246580799210,10cyclictest26-21ksoftirqd/110:40:011
246580499218,2cyclictest141rcu_preempt11:35:180
246580499218,2cyclictest141rcu_preempt10:35:120
246580499218,2cyclictest141rcu_preempt07:40:180
2465804992119,1cyclictest0-21swapper/011:26:220
2465804992119,1cyclictest0-21swapper/010:53:300
2465804992119,1cyclictest0-21swapper/010:53:300
2465804992119,1cyclictest0-21swapper/010:46:420
2465804992119,1cyclictest0-21swapper/009:55:520
2465804992119,1cyclictest0-21swapper/009:55:520
2465804992119,1cyclictest0-21swapper/009:45:170
2465804992119,1cyclictest0-21swapper/009:34:570
2465804992119,1cyclictest0-21swapper/009:26:520
2465804992119,1cyclictest0-21swapper/009:20:360
2465804992119,1cyclictest0-21swapper/008:40:140
2465804992113,6cyclictest2526403-21fw_packets10:00:140
246580499210,2cyclictest141rcu_preempt12:20:000
246580499210,20cyclictest0-21swapper/012:38:010
246580499210,20cyclictest0-21swapper/011:56:510
246580499210,20cyclictest0-21swapper/009:14:070
246580499210,20cyclictest0-21swapper/007:39:440
246580499210,20cyclictest0-21swapper/007:33:420
246580499210,20cyclictest0-21swapper/007:13:140
139099210,17rtkit-daemon2465798-21cyclictest11:00:240
2465814992017,2cyclictest3056-21PK-Backend12:18:453
2465814992017,2cyclictest3056-21PK-Backend09:23:293
246581499200,1cyclictest3056-21PK-Backend12:13:583
246581499200,1cyclictest3056-21PK-Backend12:06:483
246581499200,1cyclictest3056-21PK-Backend11:22:473
246581499200,1cyclictest3056-21PK-Backend10:20:353
246581499200,1cyclictest3056-21PK-Backend10:09:333
246581499200,1cyclictest3056-21PK-Backend08:23:203
246581499200,1cyclictest3056-21PK-Backend07:47:103
246581499200,1cyclictest3056-21PK-Backend07:39:193
246581499200,19cyclictest3056-21PK-Backend12:24:323
246581499200,19cyclictest3056-21PK-Backend11:16:123
246581499200,19cyclictest3056-21PK-Backend11:13:553
246581499200,19cyclictest3056-21PK-Backend07:53:263
246581499200,19cyclictest0-21swapper/307:22:313
246581499200,19cyclictest0-21swapper/307:11:073
246581499200,18cyclictest3056-21PK-Backend07:41:563
2465812992018,1cyclictest0-21swapper/211:03:082
2465812992018,1cyclictest0-21swapper/210:07:122
2465812992018,1cyclictest0-21swapper/209:20:402
2465812992018,1cyclictest0-21swapper/209:08:452
2465812992018,1cyclictest0-21swapper/208:29:502
2465812992017,2cyclictest0-21swapper/211:38:162
246581299200,2cyclictest0-21swapper/212:30:012
246581299200,1cyclictest0-21swapper/211:59:192
246581299200,19cyclictest0-21swapper/212:12:502
246581299200,19cyclictest0-21swapper/209:53:522
246581299200,19cyclictest0-21swapper/209:45:222
246581299200,19cyclictest0-21swapper/209:15:142
246581299200,19cyclictest0-21swapper/209:03:162
246581299200,19cyclictest0-21swapper/208:50:462
246581299200,19cyclictest0-21swapper/208:44:312
246581299200,19cyclictest0-21swapper/208:33:152
246581299200,19cyclictest0-21swapper/208:16:052
246581299200,19cyclictest0-21swapper/208:02:122
246581299200,19cyclictest0-21swapper/207:47:252
246581299200,19cyclictest0-21swapper/207:40:232
246581299200,19cyclictest0-21swapper/207:21:432
246580799208,3cyclictest26-21ksoftirqd/108:24:561
2465807992018,1cyclictest0-21swapper/111:20:081
2465807992018,1cyclictest0-21swapper/110:50:341
2465807992018,1cyclictest0-21swapper/110:50:341
2465807992018,1cyclictest0-21swapper/108:36:301
2465807992018,1cyclictest0-21swapper/108:02:161
2465807992018,1cyclictest0-21swapper/107:43:041
2465807992018,1cyclictest0-21swapper/107:21:411
2465807992017,2cyclictest0-21swapper/111:36:391
2465807992011,3cyclictest26-21ksoftirqd/107:55:091
246580799200,19cyclictest0-21swapper/112:06:591
246580799200,19cyclictest0-21swapper/111:54:431
246580799200,19cyclictest0-21swapper/110:49:191
246580799200,19cyclictest0-21swapper/110:06:131
246580799200,19cyclictest0-21swapper/108:07:581
2465804992018,1cyclictest3056-21PK-Backend10:27:070
2465804992018,1cyclictest0-21swapper/012:23:270
2465804992018,1cyclictest0-21swapper/012:06:370
2465804992018,1cyclictest0-21swapper/010:42:220
2465804992018,1cyclictest0-21swapper/010:05:040
2465804992018,1cyclictest0-21swapper/009:44:150
2465804992018,1cyclictest0-21swapper/008:30:010
2465804992018,1cyclictest0-21swapper/008:10:060
2465804992018,1cyclictest0-21swapper/007:58:440
2465804992018,1cyclictest0-21swapper/007:45:590
246580499200,19cyclictest0-21swapper/012:26:590
246580499200,19cyclictest0-21swapper/010:13:010
246580499200,19cyclictest0-21swapper/009:36:450
246580499200,19cyclictest0-21swapper/008:01:340
246580499200,19cyclictest0-21swapper/007:52:150
2465814991917,1cyclictest3056-21PK-Backend11:45:193
2465814991917,1cyclictest3056-21PK-Backend10:19:333
2465814991917,1cyclictest0-21swapper/309:01:063
2465814991917,1cyclictest0-21swapper/307:31:433
2465814991917,1cyclictest0-21swapper/307:28:193
2465814991913,4cyclictest2581282-21sshd12:37:253
246581499190,1cyclictest3056-21PK-Backend11:41:333
246581499190,1cyclictest3056-21PK-Backend11:34:433
246581499190,1cyclictest3056-21PK-Backend10:54:563
246581499190,1cyclictest3056-21PK-Backend10:54:553
246581499190,1cyclictest3056-21PK-Backend10:39:273
246581499190,1cyclictest3056-21PK-Backend09:19:513
246581499190,1cyclictest3056-21PK-Backend09:11:473
246581499190,1cyclictest3056-21PK-Backend08:35:443
246581499190,1cyclictest3056-21PK-Backend08:09:143
246581499190,18cyclictest3056-21PK-Backend12:04:243
246581499190,18cyclictest3056-21PK-Backend08:43:213
2465812991917,1cyclictest0-21swapper/211:50:472
2465812991917,1cyclictest0-21swapper/211:40:462
2465812991917,1cyclictest0-21swapper/211:10:112
2465812991917,1cyclictest0-21swapper/211:06:392
2465812991917,1cyclictest0-21swapper/210:47:122
2465812991917,1cyclictest0-21swapper/210:22:182
2465812991917,1cyclictest0-21swapper/209:31:212
2465812991912,5cyclictest2487087-21cron08:10:012
2465812991911,6cyclictest2500401-21cat08:45:182
2465812991911,6cyclictest2473417-21find07:30:132
246581299190,1cyclictest0-21swapper/210:58:362
246581299190,18cyclictest3056-21PK-Backend10:35:022
246581299190,18cyclictest0-21swapper/211:22:352
246581299190,18cyclictest0-21swapper/211:18:012
246581299190,18cyclictest0-21swapper/209:25:542
246581299190,18cyclictest0-21swapper/208:39:182
246580799192,12cyclictest2513352-21apt-get09:25:041
2465807991917,1cyclictest3056-21PK-Backend08:46:451
2465807991917,1cyclictest3056-21PK-Backend07:10:471
2465807991917,1cyclictest0-21swapper/112:17:481
2465807991917,1cyclictest0-21swapper/111:26:441
2465807991917,1cyclictest0-21swapper/111:13:191
2465807991917,1cyclictest0-21swapper/109:37:071
2465807991914,3cyclictest26-21ksoftirqd/110:20:011
2465807991913,4cyclictest26-21ksoftirqd/112:00:001
246580799190,1cyclictest0-21swapper/112:11:461
246580799190,1cyclictest0-21swapper/110:02:081
246580799190,1cyclictest0-21swapper/109:49:171
2465804991917,1cyclictest0-21swapper/011:48:190
2465804991917,1cyclictest0-21swapper/011:32:580
2465804991917,1cyclictest0-21swapper/011:20:480
2465804991917,1cyclictest0-21swapper/011:07:090
2465804991917,1cyclictest0-21swapper/010:58:030
2465804991911,6cyclictest2571492-21cut12:10:110
2465804991911,6cyclictest2495550-21sed08:30:220
246580499190,1cyclictest0-21swapper/007:16:320
246580499190,18cyclictest0-21swapper/009:00:360
246580499190,17cyclictest2523178-21sed09:50:190
2465814991816,1cyclictest3056-21PK-Backend11:05:023
2465814991816,1cyclictest3056-21PK-Backend09:44:183
2465814991816,1cyclictest3056-21PK-Backend09:36:013
2465814991816,1cyclictest3056-21PK-Backend08:30:443
2465814991815,2cyclictest3056-21PK-Backend10:49:223
246581499181,15cyclictest2530847-21chrt10:14:013
2465814991811,5cyclictest2514550-21strings09:25:183
246581499180,7cyclictest2468289-21/usr/sbin/munin07:15:123
246581499180,1cyclictest3056-21PK-Backend09:56:583
246581499180,1cyclictest3056-21PK-Backend09:56:573
246581299186,2cyclictest141rcu_preempt09:35:172
2465812991816,1cyclictest0-21swapper/209:57:582
2465812991816,1cyclictest0-21swapper/209:57:582
2465812991812,5cyclictest2541334-21cron10:44:592
2465812991810,6cyclictest886-21NetworkManager12:32:272
246580799188,8cyclictest2498765-21ntp_states08:40:191
246580799188,3cyclictest26-21ksoftirqd/110:25:251
2465807991816,1cyclictest3056-21PK-Backend08:58:341
2465807991816,1cyclictest0-21swapper/109:53:101
2465807991816,1cyclictest0-21swapper/108:15:221
2465804991816,1cyclictest0-21swapper/008:05:260
2465804991811,5cyclictest2490937-21cat08:20:130
246580499180,1cyclictest0-21swapper/009:06:500
246581499179,6cyclictest2494046-21basename08:30:013
2465814991713,2cyclictest2536145-21sh10:30:003
2465812991715,1cyclictest0-21swapper/211:47:012
2465812991714,2cyclictest3056-21PK-Backend10:15:222
2465812991714,2cyclictest2176-21JS10:27:482
2465812991713,3cyclictest978-21snmpd11:25:092
246580799177,2cyclictest141rcu_preempt09:20:181
2465807991715,1cyclictest0-21swapper/108:13:471
2465804991715,1cyclictest0-21swapper/010:20:120
2465814991614,1cyclictest3056-21PK-Backend08:14:013
2465814991613,2cyclictest3056-21PK-Backend11:51:193
246581299167,7cyclictest2526373-21df10:00:122
246581299165,2cyclictest141rcu_preempt12:05:002
2465812991614,1cyclictest3056-21PK-Backend07:50:162
2465812991614,1cyclictest0-21swapper/212:08:042
2465812991610,5cyclictest2508844-21basename09:10:162
2465807991613,2cyclictest3056-21PK-Backend07:34:251
2465807991610,4cyclictest24730702sleep107:28:581
2465804991614,1cyclictest0-21swapper/007:29:310
2465804991611,3cyclictest2489945-21rt-features08:15:190
2465814991510,3cyclictest2578218-21grep12:30:013
2465812991513,1cyclictest0-21swapper/211:34:582
2465812991513,1cyclictest0-21swapper/210:50:452
2465812991513,1cyclictest0-21swapper/210:50:442
2465812991513,1cyclictest0-21swapper/210:31:452
2465812991513,1cyclictest0-21swapper/210:25:222
2465814991412,1cyclictest3056-21PK-Backend08:15:363
246581499141,12cyclictest2545570-21/usr/sbin/munin10:55:173
246581499139,3cyclictest3056-21PK-Backend09:05:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional