You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 03:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Jul 25, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
141844423040,8sleep10-21swapper/119:05:221
141852422940,9sleep30-21swapper/319:05:243
152783621310,2sleep00-21swapper/000:15:150
146986821200,3sleep00-21swapper/021:30:140
143495721100,5sleep33056-21PK-Backend19:50:163
144488321020,4sleep20-21swapper/220:20:102
141924827540,13sleep00-21swapper/019:06:460
141816626940,22sleep20-21swapper/219:05:152
15157922500,4sleep20-21swapper/223:40:202
15242062420,3sleep20-21swapper/200:05:142
139099380,3rtkit-daemon3056-21PK-Backend00:11:533
139099370,4rtkit-daemon3056-21PK-Backend23:46:053
139099340,3rtkit-daemon3056-21PK-Backend00:32:533
15223432330,4sleep10-21swapper/100:00:161
14807202260,3sleep0161rcuc/022:00:170
139099260,4rtkit-daemon1389-21rtkit-daemon22:01:022
139099260,15rtkit-daemon3056-21PK-Backend23:20:193
139099260,15rtkit-daemon3056-21PK-Backend23:20:193
1419636992415,2cyclictest141rcu_preempt20:35:390
141963699241,3cyclictest1445250-21df_inode20:20:130
14763832230,3chrt0-21swapper/221:50:012
141963699234,7cyclictest13-21ksoftirqd/020:00:110
1419636992321,1cyclictest0-21swapper/020:42:570
1419636992316,5cyclictest436-20systemd-journal20:19:590
1419636992314,7cyclictest1532350-21perf00:30:000
14327172220,1sleep10-21swapper/119:45:131
1419646992220,1cyclictest3056-21PK-Backend22:35:353
1419646992220,1cyclictest3056-21PK-Backend21:01:393
141964699220,20cyclictest3056-21PK-Backend23:33:213
141964699220,20cyclictest3056-21PK-Backend00:16:003
1419641992220,1cyclictest0-21swapper/221:21:572
1419641992220,1cyclictest0-21swapper/220:08:432
1419641992220,1cyclictest0-21swapper/200:21:512
1419641992215,5cyclictest1439419-21munin-run20:05:002
141964199220,21cyclictest0-21swapper/221:09:542
141964199220,21cyclictest0-21swapper/219:51:002
1419638992220,1cyclictest0-21swapper/122:26:351
1419638992220,1cyclictest0-21swapper/100:38:291
1419638992220,1cyclictest0-21swapper/100:27:141
1419638992214,6cyclictest1487731-21ntp_states22:20:191
141963899220,21cyclictest0-21swapper/119:10:191
1419636992220,1cyclictest3056-21PK-Backend19:11:110
1419636992220,1cyclictest0-21swapper/021:29:410
1419636992220,1cyclictest0-21swapper/019:26:460
1419636992220,1cyclictest0-21swapper/000:33:550
1419636992219,2cyclictest0-21swapper/022:35:110
1419636992216,4cyclictest1484976-21cron22:15:000
1419636992214,6cyclictest1516491-21ls23:45:000
139099220,4rtkit-daemon1389-21rtkit-daemon20:40:292
139099220,14rtkit-daemon0-21swapper/222:44:192
139099220,14rtkit-daemon0-21swapper/200:18:042
1419646992119,1cyclictest3056-21PK-Backend23:10:163
1419646992119,1cyclictest3056-21PK-Backend22:58:133
1419646992119,1cyclictest3056-21PK-Backend21:48:033
1419646992119,1cyclictest3056-21PK-Backend21:42:433
1419646992119,1cyclictest3056-21PK-Backend21:27:563
1419646992119,1cyclictest3056-21PK-Backend20:40:303
1419646992119,1cyclictest3056-21PK-Backend20:35:113
1419646992119,1cyclictest3056-21PK-Backend20:30:233
1419646992119,1cyclictest3056-21PK-Backend19:41:093
1419646992119,1cyclictest0-21swapper/319:57:453
1419646992119,1cyclictest0-21swapper/319:21:423
141964699210,20cyclictest0-21swapper/319:14:133
1419641992119,1cyclictest0-21swapper/222:54:522
1419641992119,1cyclictest0-21swapper/222:36:192
1419641992119,1cyclictest0-21swapper/222:14:162
1419641992119,1cyclictest0-21swapper/222:09:362
1419641992119,1cyclictest0-21swapper/221:56:182
1419641992119,1cyclictest0-21swapper/221:51:312
1419641992119,1cyclictest0-21swapper/221:30:362
1419641992119,1cyclictest0-21swapper/219:47:272
1419641992119,1cyclictest0-21swapper/219:16:312
1419641992119,1cyclictest0-21swapper/200:37:462
1419641992119,1cyclictest0-21swapper/200:32:322
1419641992116,4cyclictest1485441-21cpuspeed_turbos22:15:122
141964199210,20cyclictest3056-21PK-Backend23:51:022
141964199210,20cyclictest0-21swapper/223:57:452
141964199210,20cyclictest0-21swapper/220:51:422
141964199210,20cyclictest0-21swapper/219:57:282
1419638992119,1cyclictest0-21swapper/123:41:251
1419638992119,1cyclictest0-21swapper/123:25:291
1419638992119,1cyclictest0-21swapper/123:25:291
1419638992119,1cyclictest0-21swapper/123:19:421
1419638992119,1cyclictest0-21swapper/123:03:121
1419638992119,1cyclictest0-21swapper/122:12:221
1419638992119,1cyclictest0-21swapper/122:04:251
1419638992119,1cyclictest0-21swapper/121:40:181
1419638992119,1cyclictest0-21swapper/121:18:491
1419638992119,1cyclictest0-21swapper/121:14:361
1419638992119,1cyclictest0-21swapper/121:09:161
1419638992119,1cyclictest0-21swapper/120:54:571
1419638992119,1cyclictest0-21swapper/120:41:171
1419638992119,1cyclictest0-21swapper/120:32:521
1419638992119,1cyclictest0-21swapper/120:22:111
1419638992119,1cyclictest0-21swapper/119:53:111
1419638992119,1cyclictest0-21swapper/119:20:061
1419638992119,1cyclictest0-21swapper/119:18:101
1419638992119,1cyclictest0-21swapper/100:16:331
1419638992119,1cyclictest0-21swapper/100:07:271
1419638992115,4cyclictest1509284-21idleruntime-cro23:25:001
1419638992115,4cyclictest1509284-21idleruntime-cro23:25:001
1419638992114,5cyclictest1472570-21sshd21:36:281
1419638992114,5cyclictest1458416-21sshd20:59:281
1419638992113,5cyclictest1431408-21strings19:40:161
141963899210,20cyclictest1520191-21/usr/sbin/munin23:55:171
141963899210,20cyclictest0-21swapper/123:05:331
141963899210,20cyclictest0-21swapper/121:47:011
141963899210,20cyclictest0-21swapper/121:02:401
141963899210,20cyclictest0-21swapper/100:11:051
1419636992119,1cyclictest0-21swapper/023:53:530
1419636992119,1cyclictest0-21swapper/023:36:150
1419636992119,1cyclictest0-21swapper/022:17:130
1419636992119,1cyclictest0-21swapper/021:23:330
1419636992119,1cyclictest0-21swapper/020:26:410
1419636992119,1cyclictest0-21swapper/020:26:410
1419636992119,1cyclictest0-21swapper/019:31:250
1419636992119,1cyclictest0-21swapper/000:14:210
1419636992118,2cyclictest1433153-21/usr/sbin/munin19:45:180
1419636992111,3cyclictest13-21ksoftirqd/021:02:540
141963699210,7cyclictest13-21ksoftirqd/020:45:140
141963699210,20cyclictest0-21swapper/022:46:540
141963699210,20cyclictest0-21swapper/020:51:020
141963699210,20cyclictest0-21swapper/020:31:290
14399262200,5sleep33056-21PK-Backend20:05:123
1419646992018,1cyclictest3056-21PK-Backend23:06:313
1419646992018,1cyclictest3056-21PK-Backend22:52:323
1419646992018,1cyclictest3056-21PK-Backend22:49:483
1419646992018,1cyclictest3056-21PK-Backend22:30:083
1419646992018,1cyclictest3056-21PK-Backend22:10:413
1419646992018,1cyclictest3056-21PK-Backend21:31:003
1419646992018,1cyclictest3056-21PK-Backend21:24:513
1419646992018,1cyclictest3056-21PK-Backend20:15:433
1419646992018,1cyclictest0-21swapper/320:10:233
1419646992018,1cyclictest0-21swapper/319:27:303
1419646992017,2cyclictest3056-21PK-Backend22:15:553
1419646992017,2cyclictest3056-21PK-Backend21:37:083
1419646992017,2cyclictest3056-21PK-Backend20:55:173
1419646992017,2cyclictest3056-21PK-Backend20:02:523
1419646992013,6cyclictest1520925-21awk23:55:183
141964699201,16cyclictest1510678-21python323:25:203
141964699201,16cyclictest1510678-21python323:25:193
141964699200,8cyclictest1427251-21/usr/sbin/munin19:30:113
141964699200,1cyclictest3056-21PK-Backend19:49:543
141964699200,19cyclictest3056-21PK-Backend21:55:063
141964699200,19cyclictest3056-21PK-Backend20:48:213
141964699200,18cyclictest3056-21PK-Backend23:43:283
1419641992018,1cyclictest0-21swapper/223:19:532
1419641992018,1cyclictest0-21swapper/221:14:072
1419641992018,1cyclictest0-21swapper/220:55:342
1419641992018,1cyclictest0-21swapper/220:30:472
1419641992018,1cyclictest0-21swapper/219:36:192
1419641992018,1cyclictest0-21swapper/219:23:282
1419641992018,1cyclictest0-21swapper/200:25:292
1419641992016,3cyclictest1487177-21tr22:20:132
141964199200,1cyclictest0-21swapper/223:05:332
141964199200,19cyclictest0-21swapper/223:26:282
141964199200,19cyclictest0-21swapper/223:26:282
141964199200,19cyclictest0-21swapper/223:10:542
141964199200,19cyclictest0-21swapper/219:14:012
1419638992018,1cyclictest0-21swapper/123:33:271
1419638992018,1cyclictest0-21swapper/123:13:061
1419638992018,1cyclictest0-21swapper/122:40:281
1419638992018,1cyclictest0-21swapper/121:58:161
1419638992018,1cyclictest0-21swapper/121:52:211
1419638992018,1cyclictest0-21swapper/120:02:381
1419638992018,1cyclictest0-21swapper/119:39:191
1419638992018,1cyclictest0-21swapper/119:31:071
1419638992014,4cyclictest2156-21gnome-shell21:21:091
1419638992013,6cyclictest1530146-21sed00:20:231
1419638992013,5cyclictest1519184-21ntp_kernel_err23:50:171
1419638992011,7cyclictest1470404-21ntp_states21:30:201
141963899200,1cyclictest0-21swapper/122:15:401
141963899200,19cyclictest0-21swapper/123:37:251
141963899200,19cyclictest0-21swapper/122:47:171
141963899200,19cyclictest0-21swapper/122:05:471
141963899200,19cyclictest0-21swapper/120:29:491
141963899200,19cyclictest0-21swapper/120:29:481
141963899200,19cyclictest0-21swapper/120:06:221
141963899200,19cyclictest0-21swapper/119:55:211
141963699208,4cyclictest13-21ksoftirqd/000:20:190
141963699208,10cyclictest2164-21gdbus19:21:510
1419636992018,1cyclictest0-21swapper/023:30:410
1419636992018,1cyclictest0-21swapper/023:11:550
1419636992018,1cyclictest0-21swapper/023:07:220
1419636992018,1cyclictest0-21swapper/022:24:030
1419636992018,1cyclictest0-21swapper/021:51:450
1419636992018,1cyclictest0-21swapper/021:19:550
1419636992018,1cyclictest0-21swapper/020:59:200
1419636992018,1cyclictest0-21swapper/020:11:270
1419636992018,1cyclictest0-21swapper/019:40:240
1419636992018,1cyclictest0-21swapper/000:35:440
1419636992017,2cyclictest3056-21PK-Backend19:15:230
1419636992013,6cyclictest1495528-21sed22:45:010
1419636992013,5cyclictest1510141-21cat23:25:120
1419636992013,5cyclictest1510141-21cat23:25:120
1419636992013,5cyclictest1499196-21cat22:55:130
1419636992013,5cyclictest1436732-21expr19:55:170
141963699200,1cyclictest0-21swapper/022:53:440
141963699200,19cyclictest0-21swapper/023:58:330
141963699200,19cyclictest0-21swapper/022:09:500
141963699200,19cyclictest0-21swapper/020:06:070
139099200,17rtkit-daemon3056-21PK-Backend21:35:172
139099200,15rtkit-daemon3056-21PK-Backend22:29:433
139099200,14rtkit-daemon0-21swapper/223:01:542
1419646991916,2cyclictest3056-21PK-Backend23:36:323
1419646991916,2cyclictest3056-21PK-Backend00:27:293
141964699190,1cyclictest3056-21PK-Backend22:09:193
141964699190,1cyclictest3056-21PK-Backend22:01:013
141964699190,1cyclictest3056-21PK-Backend20:50:513
141964699190,18cyclictest0-21swapper/319:19:463
1419641991917,1cyclictest0-21swapper/221:28:062
1419641991917,1cyclictest0-21swapper/220:27:092
1419641991917,1cyclictest0-21swapper/220:27:092
1419641991917,1cyclictest0-21swapper/220:11:062
1419641991911,6cyclictest436-20systemd-journal22:30:002
141964199190,1cyclictest0-21swapper/220:47:162
141964199190,18cyclictest0-21swapper/219:34:092
1419638991917,1cyclictest0-21swapper/120:12:311
1419638991917,1cyclictest0-21swapper/100:32:281
141963899190,1cyclictest0-21swapper/122:39:401
141963899190,1cyclictest0-21swapper/122:31:261
141963899190,18cyclictest0-21swapper/119:26:281
141963699199,3cyclictest13-21ksoftirqd/021:44:310
1419636991917,1cyclictest0-21swapper/021:37:120
1419636991917,1cyclictest0-21swapper/019:51:330
1419636991916,2cyclictest161rcuc/000:07:330
1419636991913,4cyclictest1507408-21sshd23:18:280
141963699190,1cyclictest0-21swapper/021:55:570
141963699190,1cyclictest0-21swapper/019:35:320
139099190,13rtkit-daemon3056-21PK-Backend22:21:223
141964699187,10cyclictest3056-21PK-Backend23:19:243
1419646991816,1cyclictest3056-21PK-Backend23:52:063
1419646991816,1cyclictest3056-21PK-Backend21:14:443
1419646991816,1cyclictest3056-21PK-Backend20:23:203
1419646991816,1cyclictest3056-21PK-Backend00:22:153
1419646991815,2cyclictest3056-21PK-Backend00:06:063
1419641991816,1cyclictest3056-21PK-Backend21:18:192
1419641991816,1cyclictest0-21swapper/220:38:042
1419641991816,1cyclictest0-21swapper/200:10:562
141963899189,7cyclictest1444600-21cron20:20:001
1419638991816,1cyclictest0-21swapper/120:47:261
1419638991811,6cyclictest26-21ksoftirqd/122:55:121
1419638991811,5cyclictest26-21ksoftirqd/120:35:121
141963699189,7cyclictest1475722-21python321:45:200
1419636991814,2cyclictest161rcuc/021:14:580
1419636991813,3cyclictest1489332-21missed_timers22:25:160
1419636991812,5cyclictest1516883-21expr23:45:110
141963699180,7cyclictest13-21ksoftirqd/021:10:010
1419646991715,1cyclictest3056-21PK-Backend22:41:033
1419646991715,1cyclictest3056-21PK-Backend00:36:083
1419641991715,1cyclictest3056-21PK-Backend22:55:392
1419641991715,1cyclictest0-21swapper/222:45:182
1419641991715,1cyclictest0-21swapper/200:03:262
141964199170,4cyclictest1444217-21users20:15:232
1419638991715,1cyclictest0-21swapper/121:26:051
1419638991711,4cyclictest1497995-21missed_timers22:50:181
141963699171,8cyclictest13-21ksoftirqd/023:20:190
141963699171,8cyclictest13-21ksoftirqd/023:20:180
1419636991711,5cyclictest1522630-21head00:00:190
1419636991711,4cyclictest1490872-21ls22:30:150
141963699170,8cyclictest13-21ksoftirqd/023:04:280
1419646991614,1cyclictest3056-21PK-Backend23:02:453
1419646991614,1cyclictest3056-21PK-Backend21:06:533
141964199168,6cyclictest1427195-21cat19:30:002
1419641991614,1cyclictest0-21swapper/223:35:272
1419641991614,1cyclictest0-21swapper/219:42:002
1419641991611,4cyclictest1459712-21grep21:00:202
1419641991611,3cyclictest1491256-21grep22:30:202
1419638991610,4cyclictest1516898-21cpuspeed_turbos23:45:121
141964699159,4cyclictest42-21ksoftirqd/321:55:003
1419646991511,3cyclictest1523431-21sh00:05:003
1419641991510,3cyclictest1509282-21dump-pmu-power23:25:002
1419641991510,3cyclictest1509282-21dump-pmu-power23:25:002
141964699149,4cyclictest3056-21PK-Backend19:36:403
141964199148,5cyclictest1517197-21cat23:45:162
1419641991412,1cyclictest3056-21PK-Backend21:40:022
141964699133,5cyclictest42-21ksoftirqd/321:15:133
141964699130,12cyclictest3056-21PK-Backend20:29:383
141964699130,12cyclictest3056-21PK-Backend20:29:383
1419641991210,1cyclictest0-21swapper/223:31:292
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional