You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 15:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Sep 05, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24538721270,3sleep20-21swapper/210:45:142
139099760,63rtkit-daemon0-21swapper/207:05:592
16855026444,14sleep30-21swapper/307:06:323
16849526343,14sleep00-21swapper/007:05:490
16850126140,14sleep10-21swapper/107:05:541
1842772560,5sleep10-21swapper/107:50:171
139099370,3rtkit-daemon3056-21PK-Backend09:54:203
1825642360,3sleep00-21swapper/007:45:190
139099360,3rtkit-daemon3056-21PK-Backend08:31:023
2387532350,5sleep234-21ksoftirqd/210:25:212
139099350,3rtkit-daemon3056-21PK-Backend08:41:003
2596942320,5sleep30-21swapper/311:25:133
2086722310,3sleep10-21swapper/109:00:141
16895199261,4cyclictest207873-21munin-run09:00:011
1808542250,4sleep33056-21PK-Backend07:40:213
16895199250,23cyclictest198173-21/usr/sbin/munin08:30:201
168954992414,7cyclictest246982-21apt-get10:50:042
168951992415,7cyclictest436-20systemd-journal07:50:001
168949992422,1cyclictest0-21swapper/011:05:390
168959992317,5cyclictest253673-21cron11:10:003
168954992319,3cyclictest0-21swapper/211:30:192
168954992315,6cyclictest186763-21cron08:00:002
168951992315,6cyclictest232345-21cron10:10:001
168949992321,1cyclictest0-21swapper/012:34:350
139099230,17rtkit-daemon3056-21PK-Backend09:26:533
139099230,15rtkit-daemon0-21swapper/208:34:182
168959992220,1cyclictest3056-21PK-Backend09:40:013
168959992220,1cyclictest3056-21PK-Backend09:05:343
168959992220,1cyclictest0-21swapper/310:05:303
168959992213,7cyclictest242561-21sed10:35:233
16895999220,20cyclictest3056-21PK-Backend08:58:453
16895999220,20cyclictest277863-21python312:15:223
168954992214,6cyclictest436-20systemd-journal07:55:002
168951992220,1cyclictest0-21swapper/110:29:171
168951992220,1cyclictest0-21swapper/109:47:121
168951992220,1cyclictest0-21swapper/109:23:131
168951992220,1cyclictest0-21swapper/109:18:061
168951992220,1cyclictest0-21swapper/109:18:061
168951992219,2cyclictest3056-21PK-Backend12:17:191
168949992220,1cyclictest0-21swapper/012:38:130
168949992220,1cyclictest0-21swapper/011:47:370
168949992220,1cyclictest0-21swapper/011:29:580
168949992220,1cyclictest0-21swapper/011:15:260
168949992220,1cyclictest0-21swapper/010:17:260
16894999220,21cyclictest0-21swapper/008:48:100
16894999220,21cyclictest0-21swapper/008:19:110
16894999220,21cyclictest0-21swapper/008:19:100
16894999220,20cyclictest0-21swapper/010:35:040
16894999220,20cyclictest0-21swapper/009:40:010
139099220,15rtkit-daemon3056-21PK-Backend07:46:073
139099220,15rtkit-daemon3056-21PK-Backend07:16:033
2609682210,4sleep2331rcuc/211:30:012
168959992119,1cyclictest3056-21PK-Backend09:38:053
168959992119,1cyclictest3056-21PK-Backend08:50:543
168959992119,1cyclictest0-21swapper/312:30:023
168959992119,1cyclictest0-21swapper/311:50:273
168959992119,1cyclictest0-21swapper/311:23:373
168959992119,1cyclictest0-21swapper/307:53:083
16895999210,19cyclictest3056-21PK-Backend09:58:273
168954992119,1cyclictest0-21swapper/209:06:422
168954992119,1cyclictest0-21swapper/208:27:082
168954992114,6cyclictest254800-21rt-features11:10:212
16895499210,20cyclictest0-21swapper/209:34:542
16895499210,1cyclictest0-21swapper/208:47:292
16895499210,1cyclictest0-21swapper/207:29:352
168951992119,1cyclictest3056-21PK-Backend11:26:221
168951992119,1cyclictest3056-21PK-Backend11:17:161
168951992119,1cyclictest0-21swapper/110:23:561
168951992119,1cyclictest0-21swapper/110:12:271
168951992119,1cyclictest0-21swapper/110:12:271
168951992119,1cyclictest0-21swapper/109:55:511
168951992119,1cyclictest0-21swapper/108:52:171
168951992119,1cyclictest0-21swapper/108:46:081
168951992119,1cyclictest0-21swapper/107:31:261
16895199210,20cyclictest3056-21PK-Backend11:02:361
16895199210,20cyclictest0-21swapper/111:43:391
16895199210,20cyclictest0-21swapper/107:29:501
16895199210,19cyclictest3056-21PK-Backend11:21:491
168949992119,1cyclictest0-21swapper/012:22:520
168949992119,1cyclictest0-21swapper/012:15:350
168949992119,1cyclictest0-21swapper/012:08:320
168949992119,1cyclictest0-21swapper/011:01:400
168949992119,1cyclictest0-21swapper/010:54:300
168949992119,1cyclictest0-21swapper/009:30:350
168949992119,1cyclictest0-21swapper/009:23:050
168949992119,1cyclictest0-21swapper/008:33:090
168949992119,1cyclictest0-21swapper/008:06:260
168949992119,1cyclictest0-21swapper/007:36:320
168949992114,5cyclictest436-20systemd-journal10:15:000
168949992114,5cyclictest436-20systemd-journal10:15:000
16894999210,20cyclictest0-21swapper/012:03:180
16894999210,20cyclictest0-21swapper/010:49:440
16894999210,20cyclictest0-21swapper/009:49:480
16894999210,20cyclictest0-21swapper/009:15:480
16894999210,20cyclictest0-21swapper/009:15:480
16894999210,20cyclictest0-21swapper/007:23:410
16894999210,19cyclictest0-21swapper/012:26:300
139099210,14rtkit-daemon3056-21PK-Backend10:28:073
2418592200,3sleep10-21swapper/110:35:141
1798962200,3sleep33056-21PK-Backend07:40:003
168959992018,1cyclictest3056-21PK-Backend10:13:073
168959992018,1cyclictest3056-21PK-Backend10:13:073
168959992018,1cyclictest3056-21PK-Backend09:11:363
168959992018,1cyclictest3056-21PK-Backend09:02:163
168959992018,1cyclictest3056-21PK-Backend08:35:263
168959992018,1cyclictest0-21swapper/312:26:373
168959992018,1cyclictest0-21swapper/312:20:353
168959992018,1cyclictest0-21swapper/308:08:363
16895999201,7cyclictest284700-21ntp_states12:35:203
16895999200,6cyclictest275072-21latency_hist12:10:003
16895999200,1cyclictest0-21swapper/311:31:213
16895999200,19cyclictest3056-21PK-Backend08:19:513
16895999200,19cyclictest3056-21PK-Backend08:19:513
16895999200,19cyclictest3056-21PK-Backend08:11:543
16895999200,19cyclictest0-21swapper/310:48:023
16895999200,18cyclictest3056-21PK-Backend08:24:313
16895499208,9cyclictest276722-21cron12:15:002
168954992018,1cyclictest0-21swapper/212:09:062
168954992018,1cyclictest0-21swapper/211:58:382
168954992018,1cyclictest0-21swapper/211:58:382
168954992018,1cyclictest0-21swapper/211:46:212
168954992018,1cyclictest0-21swapper/210:35:102
168954992018,1cyclictest0-21swapper/209:55:082
168954992018,1cyclictest0-21swapper/209:11:492
168954992017,2cyclictest3056-21PK-Backend10:05:022
168954992014,4cyclictest34-21ksoftirqd/211:00:162
16895499200,1cyclictest0-21swapper/208:01:132
16895499200,19cyclictest0-21swapper/211:54:122
16895499200,19cyclictest0-21swapper/210:16:312
16895499200,19cyclictest0-21swapper/210:03:262
16895499200,19cyclictest0-21swapper/208:15:182
16895499200,19cyclictest0-21swapper/208:15:182
16895499200,19cyclictest0-21swapper/207:46:182
168951992018,1cyclictest3056-21PK-Backend12:23:201
168951992018,1cyclictest3056-21PK-Backend12:04:551
168951992018,1cyclictest3056-21PK-Backend11:31:431
168951992018,1cyclictest3056-21PK-Backend07:10:511
168951992018,1cyclictest0-21swapper/111:07:091
168951992018,1cyclictest0-21swapper/109:32:531
168951992018,1cyclictest0-21swapper/108:44:331
168951992018,1cyclictest0-21swapper/108:17:561
168951992018,1cyclictest0-21swapper/108:17:561
168951992018,1cyclictest0-21swapper/108:13:371
168951992018,1cyclictest0-21swapper/107:40:461
168951992018,1cyclictest0-21swapper/107:38:221
168951992018,1cyclictest0-21swapper/107:23:551
168951992017,2cyclictest3056-21PK-Backend10:43:161
168951992014,4cyclictest1095-21runrttasks11:55:091
168951992014,4cyclictest1095-21runrttasks11:55:091
168951992013,5cyclictest227150-21cron09:55:001
16895199200,1cyclictest3056-21PK-Backend12:08:541
16895199200,19cyclictest3056-21PK-Backend12:26:451
16895199200,19cyclictest0-21swapper/107:16:051
168949992018,1cyclictest0-21swapper/010:28:070
168949992018,1cyclictest0-21swapper/010:05:230
168949992018,1cyclictest0-21swapper/010:01:380
168949992018,1cyclictest0-21swapper/009:50:360
168949992018,1cyclictest0-21swapper/009:02:500
168949992018,1cyclictest0-21swapper/008:58:170
168949992018,1cyclictest0-21swapper/008:40:330
168949992018,1cyclictest0-21swapper/008:39:040
168949992018,1cyclictest0-21swapper/007:55:310
168949992018,1cyclictest0-21swapper/007:31:110
168949992014,5cyclictest217879-21rt-features09:25:210
168949992012,7cyclictest184022-21cat07:50:150
16894999200,1cyclictest0-21swapper/008:02:550
16894999200,19cyclictest0-21swapper/012:11:090
16894999200,19cyclictest0-21swapper/011:58:040
16894999200,19cyclictest0-21swapper/011:58:040
16894999200,19cyclictest0-21swapper/010:56:540
16894999200,19cyclictest0-21swapper/010:41:120
16894999200,19cyclictest0-21swapper/010:33:280
16894999200,19cyclictest0-21swapper/009:36:300
168959991917,1cyclictest3056-21PK-Backend11:42:163
168959991917,1cyclictest3056-21PK-Backend09:15:413
168959991917,1cyclictest3056-21PK-Backend09:15:413
168959991917,1cyclictest0-21swapper/312:01:363
168959991917,1cyclictest0-21swapper/311:16:413
168959991917,1cyclictest0-21swapper/308:01:263
168959991914,3cyclictest250845-21df11:00:123
168959991914,3cyclictest169908-21ntp_states07:10:203
168959991913,5cyclictest3056-21PK-Backend11:56:023
168959991913,5cyclictest3056-21PK-Backend11:56:013
16895999190,1cyclictest0-21swapper/311:13:303
16895999190,18cyclictest3056-21PK-Backend10:00:363
16895999190,18cyclictest0-21swapper/311:36:423
16895999190,18cyclictest0-21swapper/310:56:543
168954991917,1cyclictest0-21swapper/212:34:202
168954991917,1cyclictest0-21swapper/212:00:262
168954991917,1cyclictest0-21swapper/211:41:552
168954991917,1cyclictest0-21swapper/210:33:352
168954991917,1cyclictest0-21swapper/209:27:242
168954991917,1cyclictest0-21swapper/209:20:412
168954991917,1cyclictest0-21swapper/208:42:082
168954991917,1cyclictest0-21swapper/208:10:592
168954991912,5cyclictest214282-21ntpq09:15:192
168954991912,5cyclictest214282-21ntpq09:15:192
168954991912,5cyclictest189530-21ntp_states08:05:212
16895499190,1cyclictest0-21swapper/212:36:102
16895499190,1cyclictest0-21swapper/212:16:092
16895499190,1cyclictest0-21swapper/207:20:432
16895499190,18cyclictest0-21swapper/211:20:322
16895499190,18cyclictest0-21swapper/210:42:542
16895499190,18cyclictest0-21swapper/210:14:562
16895499190,18cyclictest0-21swapper/210:14:562
168951991917,1cyclictest3056-21PK-Backend11:50:281
168951991917,1cyclictest3056-21PK-Backend10:52:561
168951991917,1cyclictest3056-21PK-Backend10:46:141
168951991917,1cyclictest3056-21PK-Backend10:30:521
168951991917,1cyclictest3056-21PK-Backend08:08:441
168951991917,1cyclictest3056-21PK-Backend07:56:201
168951991917,1cyclictest0-21swapper/110:00:241
168951991917,1cyclictest0-21swapper/108:37:031
168951991917,1cyclictest0-21swapper/108:28:101
168951991913,4cyclictest26-21ksoftirqd/108:20:141
168951991912,5cyclictest211126-21cat09:05:241
168951991911,6cyclictest267118-21awk11:45:191
16895199190,1cyclictest3056-21PK-Backend08:02:081
16895199190,18cyclictest3056-21PK-Backend11:37:031
16895199190,18cyclictest231irq_work/112:30:101
168949991917,1cyclictest0-21swapper/011:52:030
168949991917,1cyclictest0-21swapper/010:22:470
168949991917,1cyclictest0-21swapper/009:11:290
168949991917,1cyclictest0-21swapper/007:17:590
168949991917,1cyclictest0-21swapper/007:10:220
168949991913,4cyclictest196423-21ntp_kernel_err08:25:200
16894999190,18cyclictest3056-21PK-Backend11:43:110
139099190,3rtkit-daemon0-21swapper/209:04:532
168959991816,1cyclictest3056-21PK-Backend10:24:163
168959991816,1cyclictest0-21swapper/312:10:353
168959991816,1cyclictest0-21swapper/307:57:343
168959991813,4cyclictest883-21dbus-daemon11:48:033
168959991813,3cyclictest239937-21df_inode10:30:173
168959991811,5cyclictest174578-21cron07:25:003
16895999180,1cyclictest3056-21PK-Backend09:49:413
168954991816,1cyclictest0-21swapper/212:20:492
168954991816,1cyclictest0-21swapper/209:54:212
168954991816,1cyclictest0-21swapper/208:37:352
168954991816,1cyclictest0-21swapper/208:23:222
168954991816,1cyclictest0-21swapper/207:11:582
168954991814,3cyclictest978-21snmpd10:57:192
168954991813,4cyclictest978-21snmpd07:34:472
168954991811,5cyclictest180502-21awk07:40:172
168951991816,1cyclictest0-21swapper/109:29:081
168951991816,1cyclictest0-21swapper/109:11:371
16895199180,16cyclictest3056-21PK-Backend10:56:411
168949991816,1cyclictest0-21swapper/008:50:330
168949991816,1cyclictest0-21swapper/008:13:290
16895999172,13cyclictest216766-21idleruntime-cro09:25:003
168959991715,1cyclictest3056-21PK-Backend07:33:553
168959991710,5cyclictest42-21ksoftirqd/310:40:123
168959991710,5cyclictest247384-21df10:50:153
168954991715,1cyclictest0-21swapper/212:26:162
168954991715,1cyclictest0-21swapper/207:18:332
168954991711,5cyclictest205510-21python308:50:222
168954991711,4cyclictest222718-21cut09:40:172
168954991710,5cyclictest252365-21grep11:05:142
16895199178,2cyclictest141rcu_preempt09:35:211
168951991715,1cyclictest3056-21PK-Backend12:37:471
168951991715,1cyclictest0-21swapper/109:43:411
168951991711,4cyclictest235305-21awk10:15:231
168949991710,6cyclictest175625-21awk07:25:210
168949991710,5cyclictest13-21ksoftirqd/009:55:230
168959991614,1cyclictest3056-21PK-Backend10:18:003
168959991614,1cyclictest3056-21PK-Backend07:27:403
16895999160,14cyclictest219959-21chrt09:31:443
16895499169,6cyclictest256077-21fw_packets11:15:152
16895499162,3cyclictest207838-21cron09:00:012
168949991612,3cyclictest181577-21latency_hist07:45:010
168949991611,3cyclictest264543-21latency_hist11:40:010
168949991610,5cyclictest254570-21grep11:10:190
168949991610,4cyclictest211603-21wc09:10:000
16894999160,14cyclictest262626-21perf11:35:000
168959991513,1cyclictest3056-21PK-Backend08:47:223
16895999150,14cyclictest3056-21PK-Backend08:27:153
168954991513,1cyclictest0-21swapper/210:20:302
168954991513,1cyclictest0-21swapper/209:46:302
168954991512,2cyclictest179835-21perf07:40:002
168954991511,2cyclictest34-21ksoftirqd/209:35:112
168954991510,3cyclictest262607-21latency_hist11:35:012
168951991513,1cyclictest3056-21PK-Backend11:10:201
168951991511,2cyclictest26-21ksoftirqd/112:11:081
168949991511,3cyclictest194433-21if_enp2s008:20:170
16894999149,3cyclictest259144-21latency_hist11:25:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional