You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 00:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Jul 08, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
46458821310,4sleep342-21ksoftirqd/309:31:143
41323227539,29sleep00-21swapper/007:08:110
139099740,59rtkit-daemon0-21swapper/207:05:112
41190226643,14sleep10-21swapper/107:05:111
41333426444,14sleep30-21swapper/307:09:313
4412292520,4sleep23056-21PK-Backend08:25:202
139099420,9rtkit-daemon3056-21PK-Backend07:10:042
413499993510,12cyclictest13-21ksoftirqd/012:25:010
5190492310,4sleep342-21ksoftirqd/312:10:013
41349999278,9cyclictest13-21ksoftirqd/010:30:130
41349999277,11cyclictest13-21ksoftirqd/009:04:410
5109932260,4chrt3056-21PK-Backend11:45:160
41350699266,10cyclictest34-21ksoftirqd/208:45:132
41349999268,9cyclictest13-21ksoftirqd/011:10:110
41350699252,16cyclictest1895073-21snapd12:29:222
413504992516,7cyclictest883-21dbus-daemon07:42:151
413499992521,3cyclictest13-21ksoftirqd/012:29:400
41349999250,3cyclictest440091-21munin-run08:25:000
413509992414,8cyclictest2164-21gdbus12:29:353
41350699248,3cyclictest4786832chrt10:14:032
413506992412,10cyclictest492667-21unin-run10:55:002
41350499241,21cyclictest437205-21df_inode08:15:141
41349999247,9cyclictest13-21ksoftirqd/009:20:110
41349999247,7cyclictest13-21ksoftirqd/008:55:200
41349999247,6cyclictest13-21ksoftirqd/008:25:130
413499992420,3cyclictest161rcuc/009:25:560
413499992417,5cyclictest496137-21systemd-run11:05:000
413499992417,4cyclictest13-21ksoftirqd/008:02:160
41350699230,4cyclictest509154-21diskstats11:40:142
41350699230,4cyclictest509154-21diskstats11:40:142
41350499230,21cyclictest3056-21PK-Backend12:33:281
41349999235,7cyclictest13-21ksoftirqd/008:05:170
41349999232,7cyclictest13-21ksoftirqd/007:50:100
413499992320,2cyclictest161rcuc/010:58:510
413499992319,3cyclictest161rcuc/009:33:100
413499992319,3cyclictest161rcuc/008:42:530
413499992319,3cyclictest161rcuc/007:56:070
413499992315,6cyclictest497845-21perf11:10:000
413499992311,9cyclictest883-21dbus-daemon07:37:140
4467542220,4sleep30-21swapper/308:40:213
413509992215,6cyclictest417858-21expr07:20:123
413509992211,9cyclictest42-21ksoftirqd/311:15:233
41350999220,21cyclictest3056-21PK-Backend08:59:083
41350699228,3cyclictest141rcu_preempt11:35:002
413504992220,1cyclictest0-21swapper/109:07:041
413504992215,5cyclictest492644-21turbostat.cron10:55:001
41349999225,7cyclictest13-21ksoftirqd/010:10:120
41349999225,6cyclictest13-21ksoftirqd/010:20:160
41349999224,7cyclictest13-21ksoftirqd/007:45:190
41349999224,7cyclictest13-21ksoftirqd/007:30:180
413499992218,3cyclictest161rcuc/012:11:170
413499992218,3cyclictest161rcuc/010:40:000
413499992213,7cyclictest437195-21df08:15:120
41349999221,19cyclictest459450-21perf09:20:000
41349999220,8cyclictest455999-21unin-run09:10:000
41349999220,6cyclictest520908-21apt12:15:110
41349999220,21cyclictest0-21swapper/007:10:110
139099220,15rtkit-daemon3056-21PK-Backend12:11:553
139099220,15rtkit-daemon3056-21PK-Backend10:40:222
139099220,15rtkit-daemon0-21swapper/207:41:072
139099220,14rtkit-daemon3056-21PK-Backend11:07:403
139099220,14rtkit-daemon0-21swapper/208:33:562
413509992119,1cyclictest3056-21PK-Backend08:00:563
413509992119,1cyclictest0-21swapper/311:48:413
413509992119,1cyclictest0-21swapper/311:41:583
413509992119,1cyclictest0-21swapper/311:41:583
413509992119,1cyclictest0-21swapper/311:30:433
413509992119,1cyclictest0-21swapper/310:38:243
413509992118,2cyclictest3056-21PK-Backend08:06:493
413509992114,4cyclictest523770-21python312:20:213
413509992114,4cyclictest476440-21python310:05:213
413509992113,6cyclictest504017-21awk11:25:173
413509992113,6cyclictest457913-21sed09:15:013
41350999210,20cyclictest3056-21PK-Backend10:13:233
41350999210,20cyclictest0-21swapper/309:39:573
41350999210,19cyclictest3056-21PK-Backend09:45:043
41350999210,19cyclictest3056-21PK-Backend08:46:373
41350999210,19cyclictest3056-21PK-Backend08:26:513
41350999210,19cyclictest3056-21PK-Backend07:37:093
41350699218,4cyclictest34-21ksoftirqd/208:56:492
413506992119,1cyclictest0-21swapper/212:12:052
413506992117,3cyclictest34-21ksoftirqd/207:15:012
413506992114,5cyclictest527048-21sshd12:30:132
41350699210,8cyclictest447182-21chrt08:44:172
41350699210,20cyclictest3056-21PK-Backend09:20:152
41350699210,20cyclictest0-21swapper/209:17:182
413504992118,2cyclictest3056-21PK-Backend09:03:331
41350499210,19cyclictest3056-21PK-Backend11:48:401
41350499210,19cyclictest3056-21PK-Backend08:10:401
41349999214,6cyclictest13-21ksoftirqd/011:54:050
41349999213,7cyclictest13-21ksoftirqd/011:15:150
41349999213,7cyclictest13-21ksoftirqd/010:38:030
41349999212,12cyclictest13-21ksoftirqd/011:30:170
413499992119,1cyclictest0-21swapper/012:39:510
413499992119,1cyclictest0-21swapper/009:35:240
413499992117,3cyclictest161rcuc/007:16:000
41349999210,2cyclictest443852-21cat08:35:010
41349999210,20cyclictest0-21swapper/009:52:000
41349999210,20cyclictest0-21swapper/008:46:300
139099210,15rtkit-daemon3056-21PK-Backend09:43:033
139099210,14rtkit-daemon3056-21PK-Backend10:22:003
139099210,14rtkit-daemon3056-21PK-Backend10:15:453
139099210,14rtkit-daemon3056-21PK-Backend07:46:093
139099210,14rtkit-daemon0-21swapper/211:24:282
139099210,14rtkit-daemon0-21swapper/207:50:522
139099210,14rtkit-daemon0-21swapper/207:46:162
41350999207,4cyclictest42-21ksoftirqd/307:40:133
413509992018,1cyclictest3056-21PK-Backend11:58:213
413509992018,1cyclictest3056-21PK-Backend09:01:183
413509992018,1cyclictest3056-21PK-Backend08:36:513
413509992018,1cyclictest0-21swapper/308:54:153
413509992017,2cyclictest3056-21PK-Backend11:14:553
413509992017,2cyclictest3056-21PK-Backend07:25:473
413509992012,6cyclictest513103-21python311:50:203
41350999200,1cyclictest3056-21PK-Backend10:41:223
41350999200,19cyclictest3056-21PK-Backend10:53:593
41350999200,19cyclictest3056-21PK-Backend10:34:393
41350999200,19cyclictest3056-21PK-Backend09:56:073
41350999200,19cyclictest3056-21PK-Backend09:50:593
41350999200,19cyclictest3056-21PK-Backend09:18:213
41350999200,19cyclictest3056-21PK-Backend07:55:003
41350999200,19cyclictest3056-21PK-Backend07:13:363
41350699208,4cyclictest34-21ksoftirqd/209:25:562
413506992018,1cyclictest3056-21PK-Backend09:50:232
413506992018,1cyclictest0-21swapper/210:48:372
413506992018,1cyclictest0-21swapper/210:48:372
413506992018,1cyclictest0-21swapper/210:23:422
413506992018,1cyclictest0-21swapper/207:55:462
413506992017,2cyclictest0-21swapper/211:02:152
413506992015,4cyclictest978-21snmpd10:56:282
413506992015,4cyclictest978-21snmpd09:57:172
413506992012,6cyclictest476280-21ntp_kernel_err10:05:182
413506992011,2cyclictest141rcu_preempt07:30:132
41350699200,7cyclictest34-21ksoftirqd/212:20:002
41350699200,19cyclictest3056-21PK-Backend12:35:232
41350699200,19cyclictest0-21swapper/211:53:052
41350699200,19cyclictest0-21swapper/210:32:002
41350699200,19cyclictest0-21swapper/209:42:532
41350699200,19cyclictest0-21swapper/209:31:512
41350699200,19cyclictest0-21swapper/209:04:202
413504992018,1cyclictest3056-21PK-Backend12:06:381
413504992018,1cyclictest0-21swapper/111:00:401
413504992017,2cyclictest3056-21PK-Backend11:32:171
413504992015,3cyclictest1095-21runrttasks11:10:121
413504992011,7cyclictest523665-21awk12:20:191
41350499200,1cyclictest3056-21PK-Backend12:27:331
41350499200,1cyclictest0-21swapper/112:36:251
41350499200,19cyclictest3056-21PK-Backend07:18:151
41350499200,19cyclictest0-21swapper/112:15:431
41350499200,19cyclictest0-21swapper/111:16:291
41350499200,19cyclictest0-21swapper/110:25:321
41350499200,19cyclictest0-21swapper/110:18:491
41350499200,19cyclictest0-21swapper/110:07:131
41350499200,19cyclictest0-21swapper/108:56:301
41350499200,19cyclictest0-21swapper/107:55:391
41350499200,19cyclictest0-21swapper/107:30:251
41349999205,8cyclictest13-21ksoftirqd/010:04:260
41349999204,6cyclictest13-21ksoftirqd/012:00:220
413499992018,1cyclictest0-21swapper/012:30:170
413499992018,1cyclictest0-21swapper/011:55:360
413499992017,2cyclictest161rcuc/009:45:470
413499992017,2cyclictest1095-21runrttasks09:44:300
413499992016,3cyclictest161rcuc/011:40:400
413499992016,3cyclictest161rcuc/011:40:400
413499992016,3cyclictest161rcuc/011:20:180
413499992015,4cyclictest13-21ksoftirqd/008:12:170
413499992013,6cyclictest449515-21cat08:50:130
41349999200,8cyclictest3056-21PK-Backend09:55:150
41349999200,7cyclictest13-21ksoftirqd/007:40:260
41349999200,2cyclictest479887-21rt-features10:15:190
41349999200,1cyclictest3056-21PK-Backend11:39:210
41349999200,18cyclictest0-21swapper/010:46:080
41349999200,18cyclictest0-21swapper/010:46:080
139099200,14rtkit-daemon3056-21PK-Backend11:00:523
139099200,14rtkit-daemon3056-21PK-Backend07:59:503
139099200,14rtkit-daemon0-21swapper/307:32:373
139099200,13rtkit-daemon0-21swapper/308:14:573
139099200,13rtkit-daemon0-21swapper/212:20:502
139099200,13rtkit-daemon0-21swapper/207:36:522
413509991917,1cyclictest3056-21PK-Backend11:21:033
413509991917,1cyclictest3056-21PK-Backend10:45:073
413509991917,1cyclictest3056-21PK-Backend10:45:073
413509991917,1cyclictest0-21swapper/311:35:163
413509991914,3cyclictest474583-21ntp_states10:00:193
413509991911,6cyclictest460561-21sensors_temp09:20:203
41350999190,1cyclictest3056-21PK-Backend09:28:213
41350999190,18cyclictest3056-21PK-Backend12:38:363
41350999190,18cyclictest3056-21PK-Backend10:28:443
41350699192,12cyclictest510509-21apt11:45:112
413506991917,1cyclictest3056-21PK-Backend09:05:092
413506991917,1cyclictest0-21swapper/210:19:572
413506991917,1cyclictest0-21swapper/209:49:352
41350699190,6cyclictest3056-21PK-Backend07:25:142
41350699190,1cyclictest0-21swapper/212:02:112
41350699190,1cyclictest0-21swapper/208:13:572
41350699190,18cyclictest0-21swapper/209:36:592
41350699190,18cyclictest0-21swapper/209:10:492
41350699190,18cyclictest0-21swapper/208:16:212
41350699190,18cyclictest0-21swapper/208:02:282
413504991917,1cyclictest0-21swapper/111:36:021
413504991917,1cyclictest0-21swapper/111:25:341
413504991917,1cyclictest0-21swapper/110:44:521
413504991917,1cyclictest0-21swapper/109:27:051
413504991917,1cyclictest0-21swapper/109:18:271
413504991916,2cyclictest3056-21PK-Backend09:14:071
413504991913,5cyclictest493610-21awk10:55:191
413504991913,4cyclictest26-21ksoftirqd/110:35:131
413504991912,5cyclictest461232-21cut09:25:011
413504991912,5cyclictest428678-21sed07:50:161
413504991911,5cyclictest436-20systemd-journal08:00:131
41350499190,1cyclictest3056-21PK-Backend11:41:581
41350499190,1cyclictest3056-21PK-Backend11:41:571
41350499190,1cyclictest3056-21PK-Backend11:20:141
41350499190,1cyclictest3056-21PK-Backend10:45:401
41350499190,1cyclictest3056-21PK-Backend10:45:401
41350499190,1cyclictest3056-21PK-Backend09:40:511
41350499190,1cyclictest3056-21PK-Backend09:31:241
41350499190,1cyclictest3056-21PK-Backend08:51:361
41350499190,1cyclictest0-21swapper/112:14:081
41350499190,1cyclictest0-21swapper/109:55:381
41350499190,1cyclictest0-21swapper/108:43:111
41350499190,1cyclictest0-21swapper/108:23:451
41350499190,18cyclictest3056-21PK-Backend09:46:461
41350499190,18cyclictest0-21swapper/110:24:441
41350499190,18cyclictest0-21swapper/108:45:221
41350499190,18cyclictest0-21swapper/108:26:561
41350499190,18cyclictest0-21swapper/107:45:261
41349999193,9cyclictest13-21ksoftirqd/010:25:120
413499991917,1cyclictest0-21swapper/007:24:040
413499991916,2cyclictest161rcuc/008:38:180
413499991915,3cyclictest161rcuc/010:53:330
413499991914,3cyclictest161rcuc/012:05:500
41349999190,1cyclictest0-21swapper/010:06:540
139099190,13rtkit-daemon3056-21PK-Backend08:20:143
413509991816,1cyclictest0-21swapper/309:06:593
413509991815,2cyclictest442220-21dpkg08:30:013
41350999181,15cyclictest527254-21memory12:30:163
413506991816,1cyclictest0-21swapper/212:07:322
413506991816,1cyclictest0-21swapper/211:28:112
413506991816,1cyclictest0-21swapper/208:20:192
413506991811,5cyclictest514131-21cut11:55:132
41350699180,4cyclictest508278-21chrt11:37:162
41350499188,8cyclictest918-21rs:main1
413504991816,1cyclictest3056-21PK-Backend12:03:401
413504991816,1cyclictest0-21swapper/110:34:241
413504991816,1cyclictest0-21swapper/109:36:311
413504991816,1cyclictest0-21swapper/107:36:131
413504991811,5cyclictest471056-21netstat09:50:161
413504991811,5cyclictest414359-21grep07:10:181
41350499180,1cyclictest0-21swapper/110:11:331
413499991816,1cyclictest0-21swapper/009:10:360
413499991815,2cyclictest420856-21cron07:30:000
413499991815,2cyclictest161rcuc/011:26:290
139099180,12rtkit-daemon0-21swapper/208:08:292
413509991715,1cyclictest3056-21PK-Backend08:18:183
413509991715,1cyclictest0-21swapper/312:17:543
413509991715,1cyclictest0-21swapper/312:02:063
41350999170,16cyclictest492909-21apt10:55:113
413506991715,1cyclictest0-21swapper/210:00:512
413506991714,2cyclictest3056-21PK-Backend10:25:182
413506991711,4cyclictest34-21ksoftirqd/208:35:142
413506991710,2cyclictest141rcu_preempt07:24:582
41350699170,5cyclictest499554-21chrt11:14:582
41350699170,5cyclictest3056-21PK-Backend08:50:162
41350499179,6cyclictest514451-21awk11:55:161
41350499172,13cyclictest474021-21cpuspeed_turbos10:00:121
41350699165,2cyclictest34-21ksoftirqd/211:05:182
41350699160,3cyclictest141rcu_preempt11:15:192
41350499168,6cyclictest883-21dbus-daemon08:30:011
41350499168,6cyclictest444439-21entropy08:35:141
413504991610,4cyclictest419376-21sshd07:25:111
41350999156,2cyclictest141rcu_preempt07:20:003
41350699158,2cyclictest34-21ksoftirqd/210:35:202
413504991511,3cyclictest433727-21df_inode08:05:131
413504991510,3cyclictest1095-21runrttasks07:20:201
413504991412,1cyclictest3056-21PK-Backend11:50:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional