You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 16:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Jul 01, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
65453321250,2sleep20-21swapper/207:30:122
69234221140,4sleep0161rcuc/009:20:000
7032152950,2sleep00-21swapper/009:50:010
7032152950,2sleep00-21swapper/009:50:000
6457432810,7sleep10-21swapper/107:05:281
64625027651,15sleep30-21swapper/307:10:003
64476726444,13sleep00-21swapper/007:05:030
64588426141,14sleep20-21swapper/207:05:302
7141722440,2sleep20-21swapper/210:20:132
7260042420,2sleep00-21swapper/010:55:000
6635412420,4sleep13056-21PK-Backend07:55:171
6469332410,3sleep00-21swapper/007:10:120
7421682380,3sleep13056-21PK-Backend11:40:121
7093702340,3sleep00-21swapper/010:05:200
6634152310,3sleep00-21swapper/007:55:150
646368993010,6cyclictest704135-21ntp_states09:50:200
7609732280,4chrt0-21swapper/012:35:010
646368992814,3cyclictest669670-21sh08:15:010
139099280,20rtkit-daemon3056-21PK-Backend12:20:180
7228642270,4sleep30-21swapper/310:45:143
7228642270,4sleep30-21swapper/310:45:143
6682972270,5sleep13056-21PK-Backend08:10:121
646368992713,3cyclictest1095-21runrttasks08:05:160
646368992613,3cyclictest751190-21ntp_offset12:05:190
64636899260,20cyclictest0-21swapper/007:48:110
646368992520,4cyclictest0-21swapper/011:05:350
646368992512,10cyclictest762796-21latency_hist12:40:010
64636899250,24cyclictest0-21swapper/011:49:080
646378992417,4cyclictest725317-21python310:50:203
64637499240,22cyclictest725176-21ntp_kernel_pll_10:50:192
646368992419,4cyclictest0-21swapper/007:43:170
646368992418,5cyclictest0-21swapper/011:13:200
646368992418,5cyclictest0-21swapper/010:57:510
646368992410,6cyclictest883-21dbus-daemon11:01:220
64637899230,3cyclictest695209-21ntp_states09:25:203
646372992320,2cyclictest3056-21PK-Backend11:08:411
64636899232,19cyclictest0-21swapper/010:45:080
64636899232,19cyclictest0-21swapper/010:45:080
646368992316,1cyclictest0-21swapper/009:01:050
646368992311,4cyclictest671657-21/usr/sbin/munin08:20:210
7088162220,3sleep13056-21PK-Backend10:05:141
646378992220,1cyclictest3056-21PK-Backend08:48:353
646378992219,2cyclictest3056-21PK-Backend09:45:473
646378992219,2cyclictest3056-21PK-Backend09:45:473
64637899221,8cyclictest758018-21df12:25:133
64637899220,3cyclictest690420-21taskset09:14:313
64637899220,20cyclictest650444-21latency_hist07:20:013
64637899220,20cyclictest3056-21PK-Backend11:29:373
646372992220,1cyclictest3056-21PK-Backend11:53:281
646372992220,1cyclictest3056-21PK-Backend10:41:231
646372992219,2cyclictest3056-21PK-Backend09:07:261
646372992218,3cyclictest26-21ksoftirqd/109:55:001
646372992212,8cyclictest726664-21irqstats10:55:151
64636899222,7cyclictest676884-21apt08:35:130
646368992220,1cyclictest0-21swapper/012:00:030
646368992220,1cyclictest0-21swapper/011:55:580
646368992220,1cyclictest0-21swapper/011:30:560
646368992220,1cyclictest0-21swapper/010:34:530
646368992220,1cyclictest0-21swapper/009:29:510
646368992220,1cyclictest0-21swapper/009:23:280
646368992220,1cyclictest0-21swapper/009:11:320
646368992220,1cyclictest0-21swapper/009:06:190
646368992220,1cyclictest0-21swapper/008:26:230
646368992220,1cyclictest0-21swapper/007:22:090
646368992217,1cyclictest3056-21PK-Backend11:41:250
646368992210,10cyclictest682558-21irqstats08:50:170
646368992210,10cyclictest682558-21irqstats08:50:170
64636899220,20cyclictest0-21swapper/009:59:100
139099220,14rtkit-daemon3056-21PK-Backend08:00:041
64637899212,1cyclictest0-21swapper/309:31:423
646378992119,1cyclictest3056-21PK-Backend10:58:413
646378992119,1cyclictest3056-21PK-Backend08:35:523
646378992119,1cyclictest0-21swapper/312:30:413
646378992119,1cyclictest0-21swapper/312:22:293
646378992119,1cyclictest0-21swapper/311:50:323
646378992119,1cyclictest0-21swapper/311:36:473
646378992119,1cyclictest0-21swapper/310:16:023
646378992119,1cyclictest0-21swapper/310:00:483
646378992119,1cyclictest0-21swapper/309:58:523
646378992119,1cyclictest0-21swapper/308:42:493
646378992119,1cyclictest0-21swapper/307:32:173
646378992119,1cyclictest0-21swapper/307:29:353
646378992114,5cyclictest42-21ksoftirqd/310:40:103
646378992113,6cyclictest436-20systemd-journal11:44:553
64637899210,20cyclictest0-21swapper/307:51:523
64637899210,20cyclictest0-21swapper/307:51:523
64637899210,20cyclictest0-21swapper/307:24:213
64637899210,19cyclictest3056-21PK-Backend07:12:103
64637499213,15cyclictest650289-21munin-run07:20:002
646374992116,4cyclictest34-21ksoftirqd/207:45:112
646372992119,1cyclictest0-21swapper/111:32:271
646372992119,1cyclictest0-21swapper/111:29:561
646372992119,1cyclictest0-21swapper/108:56:461
646372992119,1cyclictest0-21swapper/107:18:381
646372992113,6cyclictest436-20systemd-journal07:25:001
646368992119,1cyclictest0-21swapper/010:14:250
646368992119,1cyclictest0-21swapper/009:39:300
646368992119,1cyclictest0-21swapper/009:33:430
646368992119,1cyclictest0-21swapper/008:55:510
646368992119,1cyclictest0-21swapper/008:01:420
646368992114,6cyclictest736336-21/usr/sbin/munin11:25:000
64636899210,20cyclictest0-21swapper/008:46:440
64636899210,1cyclictest0-21swapper/012:12:070
64636899210,19cyclictest0-21swapper/007:30:120
139099210,15rtkit-daemon3056-21PK-Backend10:23:441
139099210,14rtkit-daemon0-21swapper/007:35:030
646378992018,1cyclictest0-21swapper/312:02:013
646378992018,1cyclictest0-21swapper/311:10:183
646378992018,1cyclictest0-21swapper/310:26:513
646378992018,1cyclictest0-21swapper/308:09:353
646378992018,1cyclictest0-21swapper/307:49:423
646378992012,6cyclictest719740-21seq10:35:193
646378992012,6cyclictest436-20systemd-journal09:20:003
64637899200,1cyclictest0-21swapper/310:08:183
64637899200,19cyclictest3056-21PK-Backend12:39:263
64637899200,19cyclictest3056-21PK-Backend09:43:513
64637899200,19cyclictest3056-21PK-Backend08:29:023
64637899200,19cyclictest0-21swapper/312:15:403
64637899200,19cyclictest0-21swapper/311:18:023
64637899200,19cyclictest0-21swapper/311:08:553
64637899200,19cyclictest0-21swapper/307:57:323
64637899200,19cyclictest0-21swapper/307:41:163
64637899200,18cyclictest3056-21PK-Backend08:52:273
64637899200,18cyclictest3056-21PK-Backend08:52:273
64637499208,11cyclictest650810-21apt07:20:112
646374992018,1cyclictest0-21swapper/211:32:292
646374992018,1cyclictest0-21swapper/211:29:122
646374992018,1cyclictest0-21swapper/210:56:072
646374992018,1cyclictest0-21swapper/210:45:522
646374992018,1cyclictest0-21swapper/210:45:522
646374992018,1cyclictest0-21swapper/209:23:592
646374992018,1cyclictest0-21swapper/207:29:162
646374992013,5cyclictest730371-21ntpq11:05:182
646374992010,8cyclictest697295-21users09:30:362
64637499200,2cyclictest0-21swapper/209:59:432
64637499200,19cyclictest0-21swapper/212:04:192
64637499200,19cyclictest0-21swapper/210:42:002
64637499200,19cyclictest0-21swapper/210:35:382
64637499200,19cyclictest0-21swapper/210:14:222
64637499200,19cyclictest0-21swapper/208:12:142
64637499200,19cyclictest0-21swapper/208:04:502
646372992018,1cyclictest3056-21PK-Backend10:11:291
646372992016,3cyclictest26-21ksoftirqd/111:20:001
64637299200,19cyclictest0-21swapper/112:16:191
64637299200,19cyclictest0-21swapper/110:26:421
64637299200,11cyclictest26-21ksoftirqd/108:45:001
646368992018,1cyclictest3056-21PK-Backend10:26:350
646368992018,1cyclictest0-21swapper/011:53:140
646368992018,1cyclictest0-21swapper/007:52:310
646368992018,1cyclictest0-21swapper/007:52:300
646368992017,2cyclictest3056-21PK-Backend11:16:370
64636899200,1cyclictest0-21swapper/010:01:060
64636899200,19cyclictest0-21swapper/012:27:070
64636899200,19cyclictest0-21swapper/012:18:560
64636899200,19cyclictest0-21swapper/011:27:390
64636899200,19cyclictest0-21swapper/010:40:410
64636899200,19cyclictest0-21swapper/010:22:440
64636899200,19cyclictest0-21swapper/009:43:220
139099200,14rtkit-daemon3056-21PK-Backend12:30:371
139099200,14rtkit-daemon3056-21PK-Backend11:10:291
139099200,14rtkit-daemon0-21swapper/108:08:001
139099200,13rtkit-daemon0-21swapper/108:24:411
7020152190,2sleep1251rcuc/109:45:131
7020152190,2sleep1251rcuc/109:45:121
646378991917,1cyclictest0-21swapper/309:04:523
646378991915,3cyclictest2235-21evolution-calen08:11:223
646378991914,3cyclictest693057-21memory09:20:173
646378991912,5cyclictest883-21dbus-daemon11:48:043
646378991912,4cyclictest711824-21unin-run10:15:003
646378991911,6cyclictest713968-21expr10:20:133
646378991911,6cyclictest699683-21sleep09:39:593
64637899190,1cyclictest3056-21PK-Backend11:23:493
64637899190,1cyclictest3056-21PK-Backend08:31:463
64637899190,1cyclictest3056-21PK-Backend08:21:383
64637899190,1cyclictest0-21swapper/312:08:033
64637899190,1cyclictest0-21swapper/307:36:233
64637899190,17cyclictest729393-21/usr/sbin/munin11:05:003
646374991917,1cyclictest0-21swapper/212:35:092
646374991917,1cyclictest0-21swapper/212:08:252
646374991917,1cyclictest0-21swapper/211:41:012
646374991917,1cyclictest0-21swapper/211:23:582
646374991917,1cyclictest0-21swapper/210:31:472
646374991917,1cyclictest0-21swapper/210:03:342
646374991917,1cyclictest0-21swapper/208:55:332
646374991917,1cyclictest0-21swapper/208:50:212
646374991917,1cyclictest0-21swapper/208:50:202
646374991917,1cyclictest0-21swapper/208:32:422
646374991917,1cyclictest0-21swapper/208:07:342
646374991917,1cyclictest0-21swapper/207:44:292
646374991917,1cyclictest0-21swapper/207:13:542
646374991913,4cyclictest662986-21acpi07:55:122
646374991912,6cyclictest728057-21tr11:00:112
646374991912,6cyclictest688324-21sed09:05:212
646374991912,5cyclictest883-21dbus-daemon12:15:002
646374991912,5cyclictest747703-21ntpq11:55:192
646374991912,5cyclictest733133-21dpkg11:15:112
646374991912,5cyclictest713368-21sshd10:18:112
646374991911,6cyclictest757859-21cat12:25:132
64637499190,1cyclictest0-21swapper/211:52:512
64637499190,1cyclictest0-21swapper/211:11:342
64637499190,1cyclictest0-21swapper/210:25:592
64637499190,1cyclictest0-21swapper/209:41:582
64637499190,1cyclictest0-21swapper/208:45:072
64637499190,1cyclictest0-21swapper/207:53:432
64637499190,1cyclictest0-21swapper/207:53:422
64637499190,18cyclictest0-21swapper/211:39:392
64637499190,18cyclictest0-21swapper/210:08:012
64637499190,18cyclictest0-21swapper/209:52:452
64637499190,18cyclictest0-21swapper/209:36:102
64637499190,18cyclictest0-21swapper/209:28:252
646372991917,1cyclictest3056-21PK-Backend10:32:031
646372991917,1cyclictest0-21swapper/111:02:121
646372991917,1cyclictest0-21swapper/108:39:421
646372991916,2cyclictest3056-21PK-Backend12:36:541
646372991915,2cyclictest7202162sleep110:37:061
646372991915,2cyclictest6974922chrt09:32:571
646372991915,2cyclictest690466-21dump-pmu-power09:15:001
646372991914,3cyclictest7083112sleep110:04:571
64637299190,1cyclictest3056-21PK-Backend09:36:331
64637299190,1cyclictest3056-21PK-Backend09:27:481
64637299190,1cyclictest0-21swapper/112:20:111
64637299190,1cyclictest0-21swapper/108:30:501
64637299190,1cyclictest0-21swapper/108:18:261
64637299190,18cyclictest0-21swapper/111:21:171
646368991917,1cyclictest3056-21PK-Backend08:40:220
646368991917,1cyclictest0-21swapper/010:18:180
646368991912,5cyclictest436-20systemd-journal07:16:110
139099190,14rtkit-daemon3056-21PK-Backend11:57:271
646378991816,1cyclictest3056-21PK-Backend09:50:493
646378991816,1cyclictest3056-21PK-Backend08:55:453
646378991816,1cyclictest0-21swapper/312:10:143
646378991816,1cyclictest0-21swapper/311:30:593
646378991813,3cyclictest747474-21irqstats11:55:163
646378991811,5cyclictest717667-21which10:30:163
646378991810,6cyclictest688348-21awk09:05:233
64637499189,7cyclictest686616-21users09:00:212
646374991816,1cyclictest0-21swapper/212:31:032
646374991816,1cyclictest0-21swapper/208:39:182
646374991813,3cyclictest690475-21latency_hist09:15:012
646374991811,6cyclictest756631-21cat12:20:232
646374991811,5cyclictest692374-21sed09:20:022
646372991816,1cyclictest0-21swapper/112:05:451
646372991816,1cyclictest0-21swapper/108:48:411
646372991814,2cyclictest6920762sleep109:18:301
646372991813,3cyclictest725914-21dump-pmu-power10:55:001
646372991810,6cyclictest654006-21cron07:30:001
64637299180,1cyclictest0-21swapper/109:44:581
64637299180,17cyclictest3056-21PK-Backend07:51:291
64637299180,17cyclictest3056-21PK-Backend07:51:291
646368991812,5cyclictest675526-21diskstats08:30:160
64637499179,6cyclictest679193-21memory08:40:182
646374991715,1cyclictest3056-21PK-Backend08:20:262
646374991715,1cyclictest0-21swapper/211:48:112
646374991715,1cyclictest0-21swapper/208:16:202
64637299179,6cyclictest648054-21sh07:15:011
64637299177,4cyclictest26-21ksoftirqd/109:21:191
64637299177,4cyclictest26-21ksoftirqd/107:45:421
64637299174,2cyclictest141rcu_preempt08:51:341
64637299174,2cyclictest141rcu_preempt08:51:341
646372991714,2cyclictest3056-21PK-Backend09:58:441
646372991713,2cyclictest7589952sleep112:28:371
646372991713,2cyclictest7534882sleep112:13:001
646372991713,2cyclictest7412502chrt11:36:351
646372991712,3cyclictest7240992chrt10:49:041
646372991712,3cyclictest7240992chrt10:49:041
646372991712,3cyclictest6868352sleep109:03:011
64637299170,15cyclictest3056-21PK-Backend07:32:361
646368991715,1cyclictest0-21swapper/007:28:500
646368991711,4cyclictest720853-21cut10:40:000
64637899161,4cyclictest664944-21cstates08:00:133
646378991612,3cyclictest670802-21python308:15:193
64637499169,6cyclictest754675-21uname12:15:202
646374991614,1cyclictest0-21swapper/209:45:152
646374991614,1cyclictest0-21swapper/209:45:152
646374991614,1cyclictest0-21swapper/207:38:152
646374991611,4cyclictest3056-21PK-Backend08:25:252
64637299166,4cyclictest26-21ksoftirqd/111:45:181
646372991614,1cyclictest3056-21PK-Backend07:37:371
646372991611,3cyclictest7500502chrt12:03:231
646372991610,4cyclictest3056-21PK-Backend07:41:101
646368991611,3cyclictest671401-21cron08:20:000
646372991511,2cyclictest26-21ksoftirqd/108:26:481
64637299143,2cyclictest141rcu_preempt10:15:031
64636899137,4cyclictest0-21swapper/011:37:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional