You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 15:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Jul 15, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
415692221400,2sleep13056-21PK-Backend21:10:101
412115721190,2sleep20-21swapper/219:25:222
186722960,3sleep20-21swapper/223:45:182
139099760,66rtkit-daemon0-21swapper/319:08:503
411394626646,14sleep10-21swapper/119:06:131
411403026444,14sleep20-21swapper/219:07:172
411273226444,14sleep00-21swapper/019:05:020
41500392480,4sleep13056-21PK-Backend20:50:131
41223662420,3sleep30-21swapper/319:30:123
41776562310,8sleep234-21ksoftirqd/222:07:222
411434399292,5cyclictest26-21ksoftirqd/123:25:001
411434399290,6cyclictest3056-21PK-Backend00:30:011
411434399290,6cyclictest3056-21PK-Backend00:30:001
411434599271,5cyclictest37142-21apt-config00:40:013
411434599270,8cyclictest21346-21apt-config23:55:013
411434399273,2cyclictest141rcu_preempt19:15:011
411434399271,7cyclictest26-21ksoftirqd/119:45:011
411434399270,6cyclictest3056-21PK-Backend00:35:001
4114345992610,5cyclictest4161168-21awk21:20:203
411434399268,8cyclictest26-21ksoftirqd/100:08:421
411434399262,2cyclictest141rcu_preempt00:40:001
411434399260,6cyclictest3056-21PK-Backend21:50:001
411434599254,6cyclictest4187672-21python322:35:203
411434599252,5cyclictest4133362-21timerandwakeup20:00:223
411434399252,4cyclictest26-21ksoftirqd/119:29:591
411434399252,2cyclictest141rcu_preempt21:55:011
411434399251,4cyclictest26-21ksoftirqd/122:45:001
411434399250,6cyclictest3056-21PK-Backend00:25:001
411434399250,6cyclictest26-21ksoftirqd/123:05:001
411434299258,7cyclictest4118124-21sshd19:18:200
411434599241,3cyclictest4158671-21apt21:15:133
411434399242,2cyclictest141rcu_preempt22:10:011
411434399241,3cyclictest26-21ksoftirqd/123:55:001
411434399240,6cyclictest3056-21PK-Backend22:15:011
187272240,4sleep3411rcuc/323:45:203
411434599233,7cyclictest4182888-21runrttasks22:22:443
4114345992316,5cyclictest886-21NetworkManager21:00:003
411434399238,4cyclictest26-21ksoftirqd/122:05:001
411434399231,4cyclictest26-21ksoftirqd/120:10:011
411434399230,6cyclictest3056-21PK-Backend20:15:001
411434399230,6cyclictest3056-21PK-Backend19:20:201
139099230,3rtkit-daemon1389-21rtkit-daemon21:57:353
139099230,15rtkit-daemon0-21swapper/223:00:122
411434599228,9cyclictest42-21ksoftirqd/300:00:123
411434599226,5cyclictest42-21ksoftirqd/323:35:183
411434599223,7cyclictest4172521-21runrttasks21:53:093
411434599223,6cyclictest4118129-21taskset19:18:243
411434599223,4cyclictest4156226-21sed21:05:223
411434599222,8cyclictest4177795-21runrttasks22:09:093
411434599222,6cyclictest4154434-21taskset21:02:043
4114345992220,1cyclictest0-21swapper/321:27:563
4114345992216,4cyclictest27748-21python300:10:213
4114345992214,6cyclictest436-20systemd-journal20:39:543
411434599221,19cyclictest873-21python322:55:213
411434599220,21cyclictest0-21swapper/322:47:323
411434499227,13cyclictest21130-21cron23:55:012
4114344992220,1cyclictest0-21swapper/221:33:572
4114344992220,1cyclictest0-21swapper/221:33:572
4114344992220,1cyclictest0-21swapper/220:25:032
4114344992220,1cyclictest0-21swapper/220:06:092
4114344992220,1cyclictest0-21swapper/200:03:432
4114344992218,3cyclictest4169077-21systemd-veritys21:44:472
411434399224,2cyclictest141rcu_preempt23:15:011
411434399223,6cyclictest26-21ksoftirqd/123:55:221
411434399223,2cyclictest141rcu_preempt21:40:001
411434399221,8cyclictest26-21ksoftirqd/123:35:011
4114343992211,3cyclictest26-21ksoftirqd/122:18:501
4114343992210,4cyclictest26-21ksoftirqd/121:41:571
411434399220,5cyclictest3056-21PK-Backend19:56:441
411434399220,4cyclictest26-21ksoftirqd/123:00:011
4114342992220,1cyclictest0-21swapper/022:26:260
411434599215,5cyclictest42-21ksoftirqd/322:04:453
411434599213,6cyclictest3351-21runrttasks23:02:583
411434599213,5cyclictest4157675-21ntp_states21:10:203
4114345992119,1cyclictest0-21swapper/323:29:513
4114345992119,1cyclictest0-21swapper/323:29:503
4114345992119,1cyclictest0-21swapper/323:16:253
4114345992119,1cyclictest0-21swapper/322:10:563
4114345992119,1cyclictest0-21swapper/320:31:593
4114345992119,1cyclictest0-21swapper/300:30:333
4114345992119,1cyclictest0-21swapper/300:28:033
4114345992119,1cyclictest0-21swapper/300:28:033
411434599211,5cyclictest0-21swapper/323:55:133
4114345992113,6cyclictest4511-21ntpq23:05:193
4114345992113,6cyclictest4189944-21sshd22:44:233
411434599211,18cyclictest4120015-21perf19:25:013
411434599210,20cyclictest0-21swapper/323:43:013
411434599210,20cyclictest0-21swapper/322:15:553
411434599210,20cyclictest0-21swapper/300:22:293
4114344992119,1cyclictest0-21swapper/223:41:532
4114344992119,1cyclictest0-21swapper/223:26:462
4114344992119,1cyclictest0-21swapper/223:26:452
4114344992119,1cyclictest0-21swapper/222:20:272
4114344992119,1cyclictest0-21swapper/221:56:342
4114344992119,1cyclictest0-21swapper/221:06:462
4114344992119,1cyclictest0-21swapper/220:49:432
4114344992119,1cyclictest0-21swapper/220:37:332
4114344992119,1cyclictest0-21swapper/220:15:362
4114344992119,1cyclictest0-21swapper/219:51:302
4114344992119,1cyclictest0-21swapper/219:49:332
4114344992119,1cyclictest0-21swapper/219:42:032
4114344992119,1cyclictest0-21swapper/219:39:332
4114344992119,1cyclictest0-21swapper/219:32:512
4114344992113,6cyclictest25135-21cat00:05:122
411434499210,20cyclictest0-21swapper/223:35:442
411434499210,20cyclictest0-21swapper/219:16:212
411434499210,20cyclictest0-21swapper/200:36:412
411434499210,19cyclictest0-21swapper/221:50:122
411434399214,6cyclictest4174297-21chrt21:58:431
411434399214,2cyclictest141rcu_preempt19:15:011
411434399212,3cyclictest26-21ksoftirqd/120:45:001
411434399211,2cyclictest141rcu_preempt20:35:011
411434399210,7cyclictest26-21ksoftirqd/123:25:131
411434399210,7cyclictest26-21ksoftirqd/123:25:131
411434399210,6cyclictest3056-21PK-Backend21:20:011
411434299214,2cyclictest141rcu_preempt19:29:590
4114342992119,1cyclictest0-21swapper/023:50:010
4114342992119,1cyclictest0-21swapper/023:33:310
4114342992119,1cyclictest0-21swapper/023:23:050
4114342992119,1cyclictest0-21swapper/022:30:460
4114342992119,1cyclictest0-21swapper/022:02:340
4114342992119,1cyclictest0-21swapper/021:50:570
4114342992119,1cyclictest0-21swapper/021:29:410
4114342992119,1cyclictest0-21swapper/020:26:280
4114342992119,1cyclictest0-21swapper/020:09:180
4114342992119,1cyclictest0-21swapper/019:37:130
4114342992119,1cyclictest0-21swapper/019:13:560
4114342992119,1cyclictest0-21swapper/000:19:350
4114342992118,2cyclictest0-21swapper/020:53:450
4114342992116,3cyclictest4164268-21hddtemp_smartct21:30:160
4114342992116,3cyclictest4164268-21hddtemp_smartct21:30:150
4114342992113,7cyclictest4136242-21df20:10:150
411434299210,20cyclictest3056-21PK-Backend00:30:580
411434299210,20cyclictest0-21swapper/023:36:360
411434299210,20cyclictest0-21swapper/023:26:280
411434299210,20cyclictest0-21swapper/023:26:280
411434299210,20cyclictest0-21swapper/023:15:400
411434299210,20cyclictest0-21swapper/022:54:240
411434299210,20cyclictest0-21swapper/021:42:320
411434299210,20cyclictest0-21swapper/019:43:570
41889482200,4chrt0-21swapper/022:40:160
411434599207,3cyclictest4126579-21taskset19:40:383
411434599203,6cyclictest42-21ksoftirqd/321:43:173
411434599203,5cyclictest4128146-21cat19:45:243
411434599202,8cyclictest42-21ksoftirqd/319:35:363
4114345992018,1cyclictest0-21swapper/323:14:083
4114345992018,1cyclictest0-21swapper/319:12:573
4114345992018,1cyclictest0-21swapper/300:09:583
4114345992017,2cyclictest0-21swapper/323:34:363
4114345992013,5cyclictest4193846-21date22:55:003
4114345992013,4cyclictest4139350-21apt20:20:113
411434599200,19cyclictest0-21swapper/320:07:183
411434599200,19cyclictest0-21swapper/319:56:573
4114344992018,1cyclictest0-21swapper/223:20:302
4114344992018,1cyclictest0-21swapper/222:46:572
4114344992018,1cyclictest0-21swapper/222:33:252
4114344992018,1cyclictest0-21swapper/222:25:212
4114344992018,1cyclictest0-21swapper/221:10:312
4114344992018,1cyclictest0-21swapper/220:23:212
4114344992018,1cyclictest0-21swapper/220:13:532
4114344992018,1cyclictest0-21swapper/220:01:162
4114344992018,1cyclictest0-21swapper/200:16:272
4114344992015,4cyclictest978-21snmpd20:40:572
4114344992015,3cyclictest978-21snmpd20:54:472
4114344992013,5cyclictest4188647-21cat22:40:132
411434499200,1cyclictest22341-21strings23:55:182
411434499200,1cyclictest0-21swapper/221:38:362
411434499200,19cyclictest0-21swapper/222:37:442
411434499200,19cyclictest0-21swapper/222:04:252
411434499200,19cyclictest0-21swapper/221:29:182
411434499200,19cyclictest0-21swapper/220:55:452
411434499200,19cyclictest0-21swapper/200:34:452
411434399209,3cyclictest26-21ksoftirqd/120:35:341
411434399203,4cyclictest26-21ksoftirqd/122:50:151
4114343992018,1cyclictest3056-21PK-Backend22:33:301
4114343992018,1cyclictest0-21swapper/119:47:081
411434399201,4cyclictest26-21ksoftirqd/123:40:001
4114343992011,7cyclictest251rcuc/119:35:131
411434399200,1cyclictest3056-21PK-Backend20:04:201
411434399200,19cyclictest3056-21PK-Backend22:48:521
411434399200,19cyclictest3056-21PK-Backend22:23:581
411434399200,19cyclictest3056-21PK-Backend21:06:311
411434399200,10cyclictest26-21ksoftirqd/120:20:001
4114342992018,1cyclictest3056-21PK-Backend20:00:390
4114342992018,1cyclictest0-21swapper/022:10:440
4114342992018,1cyclictest0-21swapper/020:58:250
4114342992018,1cyclictest0-21swapper/020:18:100
4114342992018,1cyclictest0-21swapper/000:35:230
4114342992018,1cyclictest0-21swapper/000:13:140
4114342992018,1cyclictest0-21swapper/000:03:120
4114342992012,6cyclictest4147356-21users20:40:240
411434299200,19cyclictest0-21swapper/021:15:490
139099200,3rtkit-daemon1389-21rtkit-daemon20:12:583
411434599199,3cyclictest42-21ksoftirqd/319:50:083
4114345991917,1cyclictest0-21swapper/322:28:263
4114345991917,1cyclictest0-21swapper/321:46:213
4114345991917,1cyclictest0-21swapper/321:36:083
4114345991917,1cyclictest0-21swapper/321:32:013
4114345991917,1cyclictest0-21swapper/321:32:013
411434599191,6cyclictest42-21ksoftirqd/319:25:173
411434599190,5cyclictest0-21swapper/300:15:253
411434599190,1cyclictest0-21swapper/322:31:383
411434599190,1cyclictest0-21swapper/320:50:103
4114344991917,1cyclictest0-21swapper/222:11:082
4114344991917,1cyclictest0-21swapper/221:03:352
4114344991917,1cyclictest0-21swapper/219:55:222
4114344991917,1cyclictest0-21swapper/219:13:102
4114344991917,1cyclictest0-21swapper/200:20:202
4114344991915,3cyclictest978-21snmpd19:23:492
411434499190,1cyclictest0-21swapper/223:30:102
411434499190,1cyclictest0-21swapper/222:51:162
411434499190,1cyclictest0-21swapper/222:15:142
411434499190,1cyclictest0-21swapper/221:47:022
411434499190,1cyclictest0-21swapper/221:17:272
411434399193,6cyclictest26-21ksoftirqd/121:30:161
411434399193,6cyclictest26-21ksoftirqd/121:30:151
411434399193,5cyclictest3056-21PK-Backend21:00:191
411434399191,7cyclictest26-21ksoftirqd/119:50:121
4114343991917,1cyclictest3056-21PK-Backend20:27:041
4114343991917,1cyclictest3056-21PK-Backend19:34:521
4114343991917,1cyclictest3056-21PK-Backend00:16:251
4114343991916,2cyclictest3056-21PK-Backend00:10:171
411434399190,9cyclictest26-21ksoftirqd/123:09:311
411434399190,7cyclictest17674-21latency_hist23:45:001
411434399190,5cyclictest3056-21PK-Backend20:50:001
4114342991917,1cyclictest0-21swapper/023:43:190
4114342991917,1cyclictest0-21swapper/022:16:530
4114342991917,1cyclictest0-21swapper/021:12:390
4114342991917,1cyclictest0-21swapper/020:24:450
4114342991917,1cyclictest0-21swapper/019:45:530
4114342991917,1cyclictest0-21swapper/000:21:310
4114342991916,2cyclictest3056-21PK-Backend22:46:540
4114342991916,2cyclictest3056-21PK-Backend00:25:570
4114342991916,2cyclictest3056-21PK-Backend00:25:570
411434299190,1cyclictest0-21swapper/022:08:080
411434299190,1cyclictest0-21swapper/021:23:190
411434299190,1cyclictest0-21swapper/019:59:310
411434299190,1cyclictest0-21swapper/019:51:260
411434299190,18cyclictest0-21swapper/022:38:150
4114345991813,3cyclictest4137977-21df20:15:153
411434599180,16cyclictest4141826-21ntp_states20:25:203
411434599180,15cyclictest436-20systemd-journal20:45:013
4114344991816,1cyclictest0-21swapper/200:27:152
4114344991816,1cyclictest0-21swapper/200:27:152
4114344991816,1cyclictest0-21swapper/200:10:052
4114344991812,5cyclictest1230-21seq22:56:202
411434399188,4cyclictest26-21ksoftirqd/120:20:091
411434399184,2cyclictest141rcu_preempt23:15:011
411434399182,4cyclictest26-21ksoftirqd/122:35:191
411434399181,2cyclictest141rcu_preempt21:25:001
411434399180,6cyclictest3056-21PK-Backend21:24:191
411434399180,3cyclictest3056-21PK-Backend23:50:001
4114342991816,1cyclictest0-21swapper/023:59:000
4114342991816,1cyclictest0-21swapper/021:03:390
4114342991816,1cyclictest0-21swapper/019:33:350
4114342991815,2cyclictest3056-21PK-Backend21:59:220
4114342991814,2cyclictest13-21ksoftirqd/023:05:180
4114342991811,5cyclictest4183212-21cut22:25:000
4114342991810,2cyclictest141rcu_preempt23:00:020
411434599170,15cyclictest9375-21diskstats23:20:153
411434599170,15cyclictest4148490-21irqstats20:45:173
4114344991712,4cyclictest4144290-21/usr/sbin/munin20:35:002
4114343991715,1cyclictest3056-21PK-Backend22:27:091
411434399170,8cyclictest26-21ksoftirqd/100:03:531
411434399170,6cyclictest26-21ksoftirqd/120:55:171
411434299177,3cyclictest141rcu_preempt21:40:010
411434299175,2cyclictest141rcu_preempt21:10:010
4114342991715,1cyclictest3056-21PK-Backend00:08:460
4114342991715,1cyclictest0-21swapper/023:10:330
4114342991712,3cyclictest4144686-21tr20:35:140
4114342991711,5cyclictest19401-21perf23:50:000
411434499169,5cyclictest7417-21tr23:15:122
411434499163,11cyclictest4409-21cat23:05:192
4114344991610,5cyclictest2317-21gsd-power21:20:572
411434299167,3cyclictest141rcu_preempt20:50:010
4114342991613,2cyclictest0-21swapper/021:45:430
4114344991511,3cyclictest5992-21if_err_enp2s023:10:182
411434299155,2cyclictest141rcu_preempt23:00:120
4114342991511,2cyclictest13-21ksoftirqd/020:30:170
4114342991511,2cyclictest13-21ksoftirqd/019:20:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional