You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-25 - 19:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Wed Mar 25, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
389810921250,3sleep30-21swapper/307:10:123
400233921110,2sleep33056-21PK-Backend12:05:123
392340421040,4sleep03056-21PK-Backend08:20:140
401118521010,4sleep13056-21PK-Backend12:30:121
39798942950,2sleep20-21swapper/211:00:142
389750728141,34sleep20-21swapper/207:07:232
139099790,69rtkit-daemon0-21swapper/107:09:411
389662026342,13sleep30-21swapper/307:05:143
389758326242,13sleep00-21swapper/007:08:210
39239422440,3sleep20-21swapper/208:20:192
139099330,4rtkit-daemon894-21irqbalance11:00:201
139099310,2rtkit-daemon3056-21PK-Backend07:24:191
139099280,4rtkit-daemon1389-21rtkit-daemon08:58:081
3897864992716,8cyclictest883-21dbus-daemon12:32:382
3897856992716,7cyclictest1095-21runrttasks08:35:220
39268292240,4sleep03056-21PK-Backend08:30:130
39089192240,3sleep126-21ksoftirqd/107:40:001
3897869992420,2cyclictest0-21swapper/308:25:013
139099240,3rtkit-daemon1389-21rtkit-daemon09:48:470
3897869992314,7cyclictest957-21gdbus11:56:583
3897869992314,7cyclictest883-21dbus-daemon10:55:243
3897864992317,5cyclictest34-21ksoftirqd/211:46:252
3897864992315,6cyclictest2636-21xdg-desktop-por08:17:392
389786499230,3cyclictest3969473-21ntp_states10:30:152
3897856992316,5cyclictest3975399-21cron10:50:000
389786999229,2cyclictest3939633-21ntp_states09:05:193
3897869992217,4cyclictest42-21ksoftirqd/310:04:243
389786999221,19cyclictest3995420-21df_inode11:45:133
3897864992220,1cyclictest0-21swapper/211:23:012
3897864992220,1cyclictest0-21swapper/211:16:532
3897864992220,1cyclictest0-21swapper/208:58:162
3897864992216,5cyclictest34-21ksoftirqd/211:56:572
3897864992214,5cyclictest3975401-21cron10:50:002
389786499220,21cyclictest0-21swapper/211:52:142
389786499220,21cyclictest0-21swapper/207:27:372
3897856992212,7cyclictest2237-21gdbus08:12:370
39516282210,2sleep20-21swapper/209:40:152
3897869992116,3cyclictest42-21ksoftirqd/308:27:523
3897869992116,3cyclictest3991319-21apt11:35:013
3897869992113,6cyclictest883-21dbus-daemon12:33:263
3897869992111,8cyclictest42-21ksoftirqd/310:45:193
389786999210,20cyclictest3056-21PK-Backend10:17:293
389786999210,20cyclictest0-21swapper/312:13:213
389786999210,20cyclictest0-21swapper/308:19:403
3897864992119,1cyclictest3056-21PK-Backend12:03:172
3897864992119,1cyclictest0-21swapper/212:19:122
3897864992119,1cyclictest0-21swapper/211:32:142
3897864992119,1cyclictest0-21swapper/211:29:092
3897864992119,1cyclictest0-21swapper/210:56:582
3897864992119,1cyclictest0-21swapper/210:50:512
3897864992119,1cyclictest0-21swapper/210:50:512
3897864992119,1cyclictest0-21swapper/210:22:182
3897864992119,1cyclictest0-21swapper/207:55:442
3897864992119,1cyclictest0-21swapper/207:16:432
3897864992119,1cyclictest0-21swapper/207:12:172
3897864992113,6cyclictest436-20systemd-journal12:37:442
389786499210,20cyclictest0-21swapper/210:00:212
389786499210,20cyclictest0-21swapper/208:52:562
389786499210,20cyclictest0-21swapper/207:30:362
389786499210,20cyclictest0-21swapper/207:21:572
3897859992112,7cyclictest1-21systemd11:25:011
3897859992111,8cyclictest3933553-21latency_hist08:50:011
389785999210,20cyclictest0-21swapper/111:18:351
389785999210,19cyclictest3056-21PK-Backend11:51:131
3897856992114,5cyclictest883-21dbus-daemon08:48:030
3897856992111,8cyclictest3977136-21apt-get10:54:590
3897856992111,8cyclictest3977136-21apt-get10:54:590
389785699210,20cyclictest3056-21PK-Backend09:25:010
389785699210,20cyclictest0-21swapper/011:24:470
389785699210,20cyclictest0-21swapper/008:56:230
39521022200,4sleep30-21swapper/309:40:213
39174752200,4chrt0-21swapper/208:05:002
39170512200,5sleep03056-21PK-Backend08:00:210
389786999206,2cyclictest39137202chrt07:51:543
3897869992018,1cyclictest3056-21PK-Backend07:45:323
3897869992018,1cyclictest0-21swapper/307:31:213
3897869992013,5cyclictest42-21ksoftirqd/311:50:143
3897869992013,5cyclictest4008848-21sshd12:23:243
3897869992013,5cyclictest3984531-21date11:15:013
3897869992013,5cyclictest3980919-21wc11:05:013
3897869992013,5cyclictest3908749-21cron07:40:013
3897869992012,6cyclictest883-21dbus-daemon10:54:593
3897869992012,6cyclictest883-21dbus-daemon10:54:593
3897869992011,7cyclictest2636-21xdg-desktop-por09:28:593
389786999200,1cyclictest3056-21PK-Backend10:31:083
389786999200,1cyclictest3056-21PK-Backend09:02:263
389786999200,1cyclictest3056-21PK-Backend09:02:253
389786999200,1cyclictest3056-21PK-Backend08:49:353
389786999200,19cyclictest3056-21PK-Backend11:28:403
389786999200,19cyclictest3056-21PK-Backend09:52:133
389786999200,19cyclictest3056-21PK-Backend09:15:103
389786999200,19cyclictest3056-21PK-Backend08:56:233
389786999200,19cyclictest3056-21PK-Backend08:39:273
389786999200,19cyclictest3056-21PK-Backend08:34:073
389786999200,19cyclictest3056-21PK-Backend07:18:493
389786999200,19cyclictest0-21swapper/312:25:113
389786999200,19cyclictest0-21swapper/312:02:333
389786999200,19cyclictest0-21swapper/309:24:093
389786999200,19cyclictest0-21swapper/308:13:453
389786999200,18cyclictest3904062-21expr07:25:113
389786999200,13cyclictest42-21ksoftirqd/311:20:133
3897864992018,1cyclictest0-21swapper/210:06:022
3897864992018,1cyclictest0-21swapper/208:33:152
3897864992018,1cyclictest0-21swapper/208:11:522
3897864992014,4cyclictest436-20systemd-journal11:15:002
3897864992013,5cyclictest28650irq/123-ahci[0007:35:202
3897864992012,6cyclictest3962151-21fw_packets10:10:142
3897864992011,3cyclictest34-21ksoftirqd/208:36:372
389786499200,1cyclictest0-21swapper/212:20:132
389786499200,19cyclictest0-21swapper/211:42:152
389786499200,19cyclictest0-21swapper/210:42:062
389786499200,19cyclictest0-21swapper/210:29:152
389786499200,19cyclictest0-21swapper/210:18:132
389786499200,19cyclictest0-21swapper/209:48:182
389786499200,19cyclictest0-21swapper/209:36:142
389786499200,19cyclictest0-21swapper/208:25:312
389786499200,19cyclictest0-21swapper/207:48:342
3897859992018,1cyclictest3056-21PK-Backend09:28:301
3897859992018,1cyclictest3056-21PK-Backend08:02:171
3897859992018,1cyclictest0-21swapper/112:15:401
3897859992018,1cyclictest0-21swapper/112:06:541
3897859992018,1cyclictest0-21swapper/109:16:531
3897859992018,1cyclictest0-21swapper/107:30:211
3897859992018,1cyclictest0-21swapper/107:10:341
3897859992014,5cyclictest3967667-21netstat10:25:181
389785999200,1cyclictest3056-21PK-Backend11:59:571
389785999200,1cyclictest0-21swapper/109:46:201
389785999200,1cyclictest0-21swapper/109:35:201
389785999200,19cyclictest3056-21PK-Backend12:27:161
389785999200,19cyclictest0-21swapper/112:04:511
389785999200,19cyclictest0-21swapper/111:31:381
389785999200,19cyclictest0-21swapper/110:46:301
389785999200,19cyclictest0-21swapper/109:54:461
389785999200,19cyclictest0-21swapper/109:07:071
389785999200,19cyclictest0-21swapper/109:01:471
389785999200,19cyclictest0-21swapper/109:01:471
389785999200,19cyclictest0-21swapper/108:50:301
389785999200,19cyclictest0-21swapper/107:55:021
3897856992018,1cyclictest0-21swapper/012:19:140
3897856992018,1cyclictest0-21swapper/010:34:310
3897856992018,1cyclictest0-21swapper/008:52:380
3897856992018,1cyclictest0-21swapper/008:41:430
3897856992018,1cyclictest0-21swapper/008:06:540
3897856992018,1cyclictest0-21swapper/007:19:490
3897856992017,2cyclictest0-21swapper/009:11:030
3897856992013,5cyclictest4008020-21cut12:20:160
3897856992013,5cyclictest4002482-21cat12:05:150
3897856992012,6cyclictest13-21ksoftirqd/009:59:590
389785699201,1cyclictest0-21swapper/011:26:500
389785699201,17cyclictest1970-21snapd12:00:360
389785699200,1cyclictest0-21swapper/010:19:370
389785699200,19cyclictest3056-21PK-Backend12:10:290
389785699200,19cyclictest0-21swapper/011:42:440
389785699200,19cyclictest0-21swapper/010:55:540
389785699200,19cyclictest0-21swapper/010:28:370
389785699200,19cyclictest0-21swapper/009:51:310
389785699200,19cyclictest0-21swapper/009:38:410
389785699200,19cyclictest0-21swapper/009:00:550
389785699200,19cyclictest0-21swapper/009:00:550
389785699200,19cyclictest0-21swapper/007:31:470
389785699200,19cyclictest0-21swapper/007:25:390
389786999197,11cyclictest2174-21JS10:27:483
3897869991917,1cyclictest3056-21PK-Backend10:41:423
3897869991917,1cyclictest0-21swapper/309:33:343
3897869991917,1cyclictest0-21swapper/308:05:063
3897869991914,3cyclictest3915980-21apt08:00:113
389786999190,1cyclictest3056-21PK-Backend11:19:403
389786999190,1cyclictest3056-21PK-Backend09:48:503
389786999190,1cyclictest3056-21PK-Backend08:54:083
389786999190,1cyclictest3056-21PK-Backend07:20:453
389786999190,1cyclictest0-21swapper/311:43:333
389786999190,1cyclictest0-21swapper/309:55:383
389786999190,1cyclictest0-21swapper/307:43:093
389786999190,18cyclictest3056-21PK-Backend12:18:143
389786999190,18cyclictest0-21swapper/312:38:503
3897864991917,1cyclictest0-21swapper/209:56:572
3897864991917,1cyclictest0-21swapper/209:50:082
3897864991917,1cyclictest0-21swapper/209:05:592
3897864991917,1cyclictest0-21swapper/207:43:132
3897864991913,5cyclictest34-21ksoftirqd/212:27:372
3897864991911,6cyclictest3918455-21ntpq08:05:202
3897864991911,3cyclictest34-21ksoftirqd/209:30:122
389786499190,1cyclictest0-21swapper/210:37:112
389786499190,18cyclictest3056-21PK-Backend12:05:202
3897859991917,1cyclictest3056-21PK-Backend09:31:071
3897859991917,1cyclictest3056-21PK-Backend07:42:451
3897859991917,1cyclictest0-21swapper/110:40:491
3897859991917,1cyclictest0-21swapper/110:33:061
3897859991917,1cyclictest0-21swapper/110:17:241
3897859991917,1cyclictest0-21swapper/108:40:241
3897859991917,1cyclictest0-21swapper/107:47:181
3897859991916,2cyclictest3056-21PK-Backend08:16:521
3897859991913,4cyclictest26-21ksoftirqd/108:35:131
3897859991912,5cyclictest3945057-21grep09:20:211
389785999190,1cyclictest3056-21PK-Backend10:54:011
389785999190,1cyclictest3056-21PK-Backend10:54:011
389785999190,1cyclictest3056-21PK-Backend07:52:381
389785999190,1cyclictest0-21swapper/108:22:461
389785999190,1cyclictest0-21swapper/108:13:071
389785999190,1cyclictest0-21swapper/107:29:331
389785999190,18cyclictest3056-21PK-Backend12:38:511
389785999190,18cyclictest0-21swapper/112:10:471
389785999190,18cyclictest0-21swapper/110:36:571
389785999190,18cyclictest0-21swapper/107:18:381
389785699192,11cyclictest3995050-21apt11:45:110
3897856991917,1cyclictest0-21swapper/012:38:010
3897856991917,1cyclictest0-21swapper/011:50:430
3897856991917,1cyclictest0-21swapper/010:00:590
3897856991917,1cyclictest0-21swapper/009:17:390
3897856991917,1cyclictest0-21swapper/007:46:000
3897856991917,1cyclictest0-21swapper/007:37:420
3897856991916,2cyclictest3056-21PK-Backend07:55:190
3897856991915,3cyclictest3899505-21cron07:15:000
3897856991913,5cyclictest3939766-21awk09:05:210
389785699190,1cyclictest0-21swapper/012:27:130
389785699190,1cyclictest0-21swapper/011:35:490
389785699190,1cyclictest0-21swapper/011:06:220
389785699190,18cyclictest0-21swapper/009:41:180
389786999189,7cyclictest886-21NetworkManager08:43:393
3897869991814,3cyclictest978-21snmpd11:08:343
3897869991813,3cyclictest3962120-21df10:10:123
3897869991813,3cyclictest3961227-21idleruntime-cro10:10:003
3897869991810,6cyclictest3992162-21memory11:35:163
389786499189,7cyclictest883-21dbus-daemon09:29:002
3897864991816,1cyclictest0-21swapper/207:52:322
3897864991810,3cyclictest34-21ksoftirqd/211:35:112
3897859991813,3cyclictest3951158-21cpuspeed09:40:111
3897859991810,6cyclictest3988101-21apt11:25:101
389785999180,1cyclictest3056-21PK-Backend11:41:261
389785699182,12cyclictest3989850-21apt11:30:110
389785699182,12cyclictest3970468-21apt10:35:120
389785699182,11cyclictest3980881-21idleruntime-cro11:05:000
3897856991816,1cyclictest0-21swapper/009:31:040
3897856991812,5cyclictest13-21ksoftirqd/008:20:000
3897856991811,5cyclictest436-20systemd-journal11:15:540
389785699180,1cyclictest0-21swapper/010:12:070
3897869991712,3cyclictest3942110-21idleruntime-cro09:15:003
3897864991715,1cyclictest0-21swapper/209:14:592
3897864991712,3cyclictest3942535-21cpuspeed_turbos09:15:122
389786499171,13cyclictest3937159-21apt09:00:122
389786499171,13cyclictest3937159-21apt09:00:122
389786499171,11cyclictest4003497-21apt-get12:10:042
3897864991710,5cyclictest3982080-21cat11:05:202
3897859991715,1cyclictest3056-21PK-Backend11:37:331
3897859991715,1cyclictest0-21swapper/110:09:061
3897859991714,2cyclictest3056-21PK-Backend12:23:241
3897859991714,2cyclictest3056-21PK-Backend09:57:231
3897859991712,3cyclictest3995630-21missed_timers11:45:171
389785699179,5cyclictest13-21ksoftirqd/010:40:090
389785699172,5cyclictest141rcu_preempt11:55:160
3897856991715,1cyclictest0-21swapper/008:27:030
3897856991715,1cyclictest0-21swapper/007:52:560
3897856991711,5cyclictest3909823-21proc_pri07:40:180
3897856991711,4cyclictest3982604-21cron11:14:540
3897856991711,4cyclictest3959929-21tr10:05:120
3897869991613,2cyclictest3056-21PK-Backend10:36:013
3897869991610,4cyclictest3967486-21cut10:25:163
389786999160,15cyclictest0-21swapper/309:35:113
389786499163,9cyclictest3944108-21apt09:20:132
389786499162,11cyclictest82332-21update-manager08:43:412
3897859991614,1cyclictest3056-21PK-Backend11:05:291
3897859991614,1cyclictest3056-21PK-Backend10:04:261
3897859991614,1cyclictest3056-21PK-Backend08:06:381
3897859991614,1cyclictest0-21swapper/111:10:361
389785699168,6cyclictest3902476-21cat07:20:170
3897856991614,1cyclictest0-21swapper/012:30:170
3897856991614,1cyclictest0-21swapper/010:23:420
389786999151,5cyclictest141rcu_preempt10:20:183
389786499153,8cyclictest3931934-21apt08:45:112
389785999159,4cyclictest3962602-21cut10:10:191
389785999151,11cyclictest3965260-21apt10:20:141
3897859991510,4cyclictest3927561-21smartctl08:30:211
3897856991510,3cyclictest1095-21runrttasks09:23:290
389785999149,3cyclictest3941440-21processes09:10:191
389785999148,4cyclictest3926545-21grep08:30:001
389785999139,3cyclictest3056-21PK-Backend10:56:581
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional