You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 07:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sat Aug 29, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11572022580,6sleep20-21swapper/219:05:192
18588321230,2sleep20-21swapper/222:25:132
18146321050,4sleep13056-21PK-Backend22:10:201
16284521000,2sleep10-21swapper/121:20:001
11645527338,9sleep30-21swapper/319:07:173
11633527339,9sleep00-21swapper/019:05:420
11643826343,14sleep10-21swapper/119:07:041
1672122520,4sleep03056-21PK-Backend21:30:180
2326152500,5sleep00-21swapper/000:40:000
1492502500,4sleep00-21swapper/020:40:130
1228332470,2sleep20-21swapper/219:25:132
2194152450,3sleep13056-21PK-Backend00:00:191
1613032410,3sleep00-21swapper/021:15:010
116800993210,20cyclictest155608-21cron20:59:592
11680099318,13cyclictest2057592processes23:20:212
116800993012,16cyclictest160815-21latency_hist21:15:012
11680099293,21cyclictest978-21snmpd23:05:232
11680099289,17cyclictest131272-21cut19:50:012
116800992811,15cyclictest133838-21sh19:55:192
11680099277,17cyclictest185534-21latency_hist22:25:012
116800992710,15cyclictest140712-21if_enp2s020:15:162
11680099268,6cyclictest135288-21cat20:00:152
11680099267,5cyclictest230527-21sshd00:32:382
11680099267,4cyclictest199483-21basename23:05:002
11680099261,6cyclictest132221-21python319:50:212
11680099261,5cyclictest181265-21python322:10:202
116800992611,14cyclictest139417-21sshd20:10:502
1883262250,3sleep13056-21PK-Backend22:30:221
1883262250,3sleep13056-21PK-Backend22:30:211
116803992513,10cyclictest136319-21unin-run20:05:013
11680099259,14cyclictest218633-21latency_hist00:00:002
11680099259,14cyclictest1095-21runrttasks22:07:352
11680099258,3cyclictest1095-21runrttasks23:50:202
11680099258,16cyclictest2019-21gmain21:29:592
11680099258,15cyclictest163215-21tr21:20:112
11680099257,3cyclictest1095-21runrttasks21:19:072
11680099252,3cyclictest0-21swapper/222:35:112
11680099251,8cyclictest219003-21cpuspeed_turbos00:00:132
11680099251,4cyclictest155889-21apt21:00:102
11680099251,22cyclictest126305-21cpuspeed_turbos19:35:112
11680099251,11cyclictest190807-21apt22:40:112
11680099250,7cyclictest136677-21cpuspeed_turbos20:05:092
11680099250,3cyclictest150436-21awk20:45:002
116803992412,10cyclictest205666-21ntp_states23:20:193
11680099248,3cyclictest201669-21cstates23:10:122
11680099248,14cyclictest227758-21expr00:25:132
11680099248,14cyclictest227758-21expr00:25:122
11680099248,14cyclictest169001-21ntp_kernel_pll_21:35:192
11680099246,5cyclictest197642-21apt-key23:00:002
11680099246,3cyclictest203692-21kernelversion23:15:172
11680099245,5cyclictest208980-21grep23:30:172
11680099245,5cyclictest208980-21grep23:30:162
11680099241,8cyclictest193408-21python322:45:232
116800992418,5cyclictest0-21swapper/221:46:412
11680099241,6cyclictest231664-21needreboot00:35:182
11680099241,5cyclictest220992-21hddtemp_smartct00:05:152
11680099241,5cyclictest0-21swapper/221:05:112
11680099240,9cyclictest152799-21chrt20:50:142
116795992414,8cyclictest129350-21cron19:44:591
2313732230,4chrt0-21swapper/300:35:143
11680099237,3cyclictest978-21snmpd00:14:402
11680099237,14cyclictest1751832chrt21:55:092
11680099233,5cyclictest143851-21ls20:25:122
11680099232,7cyclictest182305-21expr22:15:132
11680099232,6cyclictest2156-21gnome-shell20:32:082
116800992321,1cyclictest0-21swapper/219:33:592
116800992321,1cyclictest0-21swapper/219:33:592
11680099231,8cyclictest187486-21cpuspeed_turbos22:30:132
11680099231,8cyclictest187486-21cpuspeed_turbos22:30:122
11680099231,7cyclictest195421-21taskset22:51:102
11680099231,7cyclictest178589-21cat22:05:002
11680099231,7cyclictest170878-21smartctl21:40:212
11680099231,7cyclictest167022-21irqstats21:30:162
11680099231,7cyclictest151432-21/usr/sbin/munin20:45:182
11680099231,6cyclictest227382-21latency_hist00:25:002
11680099231,5cyclictest436-20systemd-journal20:40:002
11680099231,5cyclictest142026-21apt20:20:112
11680099230,5cyclictest0-21swapper/223:45:182
116795992319,2cyclictest26-21ksoftirqd/120:49:121
139099220,15rtkit-daemon3056-21PK-Backend22:23:481
139099220,14rtkit-daemon3056-21PK-Backend20:23:151
11680399226,14cyclictest217126-21cpu23:55:133
11680099225,3cyclictest0-21swapper/223:26:582
11680099220,7cyclictest224948-21grep00:15:202
11680099220,7cyclictest211121-21awk23:35:212
11680099220,7cyclictest119949-21ntpq19:15:182
116795992220,1cyclictest3056-21PK-Backend23:11:261
116795992220,1cyclictest3056-21PK-Backend20:52:301
116795992215,4cyclictest1095-21runrttasks00:10:241
116795992213,7cyclictest436-20systemd-journal00:10:001
116795992211,2cyclictest141rcu_preempt21:29:541
116792992220,1cyclictest0-21swapper/023:41:200
116792992220,1cyclictest0-21swapper/023:00:180
116792992220,1cyclictest0-21swapper/022:53:480
116792992220,1cyclictest0-21swapper/021:02:360
116792992220,1cyclictest0-21swapper/020:52:070
116792992220,1cyclictest0-21swapper/019:58:210
116792992220,1cyclictest0-21swapper/019:48:070
139099210,15rtkit-daemon3056-21PK-Backend22:52:151
139099210,14rtkit-daemon3056-21PK-Backend23:20:391
139099210,14rtkit-daemon3056-21PK-Backend19:50:571
139099210,13rtkit-daemon0-21swapper/022:12:410
1178732210,3sleep33056-21PK-Backend19:10:343
116803992119,1cyclictest0-21swapper/321:34:333
116803992119,1cyclictest0-21swapper/321:12:023
116803992117,3cyclictest42-21ksoftirqd/323:20:003
116803992113,6cyclictest139001-21ntp_kernel_pll_20:10:183
11680399211,17cyclictest181264-21python322:10:203
11680399210,20cyclictest3056-21PK-Backend19:16:233
11680399210,20cyclictest0-21swapper/322:56:323
11680399210,20cyclictest0-21swapper/322:52:123
11680399210,19cyclictest3056-21PK-Backend21:57:383
11680399210,19cyclictest3056-21PK-Backend19:32:243
11680399210,19cyclictest3056-21PK-Backend19:32:243
116800992119,1cyclictest0-21swapper/219:21:492
11680099211,8cyclictest212286-21/usr/sbin/munin23:40:142
11680099210,6cyclictest173480-21apt21:50:112
11680099210,19cyclictest0-21swapper/219:11:002
116795992119,1cyclictest3056-21PK-Backend21:35:281
116795992119,1cyclictest3056-21PK-Backend19:56:391
116795992119,1cyclictest0-21swapper/120:37:071
116795992119,1cyclictest0-21swapper/119:26:311
116795992119,1cyclictest0-21swapper/100:15:351
11679599210,19cyclictest191539-21ntpq22:40:181
116792992119,1cyclictest0-21swapper/023:54:460
116792992119,1cyclictest0-21swapper/023:47:150
116792992119,1cyclictest0-21swapper/023:20:310
116792992119,1cyclictest0-21swapper/023:19:440
116792992119,1cyclictest0-21swapper/022:40:230
116792992119,1cyclictest0-21swapper/022:34:220
116792992119,1cyclictest0-21swapper/022:34:220
116792992119,1cyclictest0-21swapper/021:56:100
116792992119,1cyclictest0-21swapper/021:53:400
116792992119,1cyclictest0-21swapper/021:14:250
116792992119,1cyclictest0-21swapper/020:58:030
116792992119,1cyclictest0-21swapper/019:51:310
116792992119,1cyclictest0-21swapper/019:32:320
116792992119,1cyclictest0-21swapper/019:32:310
116792992119,1cyclictest0-21swapper/000:15:210
116792992119,1cyclictest0-21swapper/000:10:470
116792992113,6cyclictest436-20systemd-journal19:25:000
116792992113,6cyclictest1-21systemd19:15:020
11679299210,2cyclictest145492-21apt20:30:110
11679299210,20cyclictest0-21swapper/023:55:340
11679299210,20cyclictest0-21swapper/022:28:210
11679299210,20cyclictest0-21swapper/021:36:220
11679299210,20cyclictest0-21swapper/020:22:330
11679299210,20cyclictest0-21swapper/000:05:260
139099200,14rtkit-daemon3056-21PK-Backend23:02:281
139099200,14rtkit-daemon0-21swapper/022:57:010
139099200,12rtkit-daemon0-21swapper/020:16:550
116803992018,1cyclictest3056-21PK-Backend23:34:183
116803992018,1cyclictest3056-21PK-Backend23:34:183
116803992018,1cyclictest3056-21PK-Backend22:48:063
116803992018,1cyclictest3056-21PK-Backend21:50:543
116803992018,1cyclictest3056-21PK-Backend21:44:133
116803992018,1cyclictest0-21swapper/323:50:253
116803992018,1cyclictest0-21swapper/323:48:503
116803992018,1cyclictest0-21swapper/322:40:103
116803992018,1cyclictest0-21swapper/322:35:303
116803992018,1cyclictest0-21swapper/322:34:083
116803992018,1cyclictest0-21swapper/322:34:083
116803992018,1cyclictest0-21swapper/322:28:063
116803992018,1cyclictest0-21swapper/322:19:213
116803992018,1cyclictest0-21swapper/321:05:403
116803992018,1cyclictest0-21swapper/321:03:303
116803992018,1cyclictest0-21swapper/320:17:133
116803992018,1cyclictest0-21swapper/300:27:383
116803992018,1cyclictest0-21swapper/300:27:383
116803992017,2cyclictest3056-21PK-Backend20:31:403
116803992017,2cyclictest0-21swapper/323:11:533
116803992014,4cyclictest42-21ksoftirqd/322:05:013
11680399200,1cyclictest0-21swapper/320:05:253
11680399200,19cyclictest3056-21PK-Backend21:38:393
11680399200,19cyclictest3056-21PK-Backend00:06:283
11680399200,19cyclictest0-21swapper/323:04:233
11680399200,19cyclictest0-21swapper/320:55:123
11680399200,19cyclictest0-21swapper/319:55:363
11680399200,18cyclictest3056-21PK-Backend00:18:313
11680099200,6cyclictest129088-21chrt19:41:412
11679599203,12cyclictest119228-21apt19:15:111
116795992018,1cyclictest3056-21PK-Backend20:12:341
116795992018,1cyclictest3056-21PK-Backend20:02:141
116795992018,1cyclictest3056-21PK-Backend19:45:091
116795992018,1cyclictest0-21swapper/123:47:571
116795992018,1cyclictest0-21swapper/121:52:321
116795992018,1cyclictest0-21swapper/121:47:461
116795992018,1cyclictest0-21swapper/121:42:241
116795992018,1cyclictest0-21swapper/119:10:161
116795992017,2cyclictest3056-21PK-Backend20:41:131
116795992015,4cyclictest208155-21systemd-run23:30:001
116795992014,5cyclictest26-21ksoftirqd/122:00:171
116795992013,6cyclictest124904-21awk19:30:171
116795992013,6cyclictest124904-21awk19:30:171
116795992012,7cyclictest208783-21awk23:30:141
116795992012,7cyclictest208783-21awk23:30:131
116795992012,6cyclictest26-21ksoftirqd/123:58:431
116795992012,6cyclictest26-21ksoftirqd/123:35:191
116795992012,2cyclictest26-21ksoftirqd/121:15:011
11679599200,2cyclictest3056-21PK-Backend22:05:221
11679599200,1cyclictest3056-21PK-Backend20:05:251
11679599200,1cyclictest3056-21PK-Backend00:35:571
11679599200,19cyclictest3056-21PK-Backend23:06:461
11679599200,19cyclictest0-21swapper/123:41:281
11679599200,19cyclictest0-21swapper/121:56:451
116792992018,1cyclictest0-21swapper/022:20:220
116792992018,1cyclictest0-21swapper/021:27:370
116792992018,1cyclictest0-21swapper/019:25:210
116792992015,3cyclictest148682-21sh20:40:000
11679299200,1cyclictest0-21swapper/022:48:540
11679299200,19cyclictest3056-21PK-Backend19:40:430
11679299200,19cyclictest0-21swapper/022:35:440
11679299200,19cyclictest0-21swapper/020:10:520
11679299200,19cyclictest0-21swapper/019:13:330
11679299200,19cyclictest0-21swapper/000:00:070
1646782190,9sleep013-21ksoftirqd/021:25:010
116803991917,1cyclictest3056-21PK-Backend00:20:073
116803991917,1cyclictest3056-21PK-Backend00:11:483
116803991917,1cyclictest0-21swapper/322:08:063
116803991917,1cyclictest0-21swapper/319:20:423
116803991914,4cyclictest42-21ksoftirqd/321:24:593
116803991914,3cyclictest219521-21cut00:00:223
116803991913,4cyclictest173210-21apt21:49:593
116803991913,4cyclictest162204-21/usr/sbin/munin21:15:193
116803991912,5cyclictest153873-21cron20:54:593
11680399190,1cyclictest3056-21PK-Backend19:28:263
11680399190,1cyclictest0-21swapper/322:24:003
11680399190,1cyclictest0-21swapper/320:48:303
11680399190,1cyclictest0-21swapper/319:53:473
11680399190,1cyclictest0-21swapper/300:32:093
11680399190,18cyclictest3056-21PK-Backend23:29:103
11680399190,18cyclictest0-21swapper/323:05:113
11680399190,18cyclictest0-21swapper/320:40:393
11679599192,12cyclictest126163-21apt19:35:111
116795991917,1cyclictest3056-21PK-Backend22:28:011
116795991911,6cyclictest26-21ksoftirqd/121:30:501
116795991911,6cyclictest26-21ksoftirqd/120:30:591
116795991911,6cyclictest26-21ksoftirqd/120:15:121
116795991911,6cyclictest26-21ksoftirqd/100:30:201
11679599190,1cyclictest3056-21PK-Backend23:54:121
11679599190,1cyclictest3056-21PK-Backend22:56:201
11679599190,1cyclictest3056-21PK-Backend21:03:251
11679599190,1cyclictest0-21swapper/120:28:231
11679599190,1cyclictest0-21swapper/100:20:421
11679599190,18cyclictest3056-21PK-Backend22:19:221
116792991917,1cyclictest3056-21PK-Backend20:00:100
116792991917,1cyclictest0-21swapper/022:00:020
116792991917,1cyclictest0-21swapper/021:41:350
116792991917,1cyclictest0-21swapper/020:49:100
116792991917,1cyclictest0-21swapper/019:37:320
116792991913,5cyclictest226184-21forks00:20:140
11679299190,1cyclictest0-21swapper/023:09:300
11679299190,1cyclictest0-21swapper/022:05:500
11679299190,18cyclictest0-21swapper/023:10:170
116803991816,1cyclictest0-21swapper/319:49:353
116803991811,5cyclictest165185-21df21:25:143
116803991811,5cyclictest143475-21cron20:24:593
11680399180,1cyclictest3056-21PK-Backend23:42:493
116795991816,1cyclictest3056-21PK-Backend21:09:321
116795991816,1cyclictest3056-21PK-Backend00:28:191
116795991816,1cyclictest3056-21PK-Backend00:28:191
116795991815,2cyclictest3056-21PK-Backend21:21:361
116795991810,6cyclictest26-21ksoftirqd/123:20:011
116795991810,6cyclictest26-21ksoftirqd/122:37:061
11679599180,17cyclictest0-21swapper/119:21:101
116792991816,1cyclictest0-21swapper/023:34:520
116792991816,1cyclictest0-21swapper/023:34:520
116792991816,1cyclictest0-21swapper/021:45:480
116792991811,5cyclictest210623-21cat23:35:150
11679299180,1cyclictest0-21swapper/021:06:080
116803991715,1cyclictest0-21swapper/319:44:353
116803991714,2cyclictest3056-21PK-Backend19:37:253
116803991712,4cyclictest42-21ksoftirqd/320:40:013
116803991712,2cyclictest42-21ksoftirqd/323:35:213
116795991715,1cyclictest3056-21PK-Backend20:55:131
116795991710,5cyclictest26-21ksoftirqd/122:45:171
11679299172,13cyclictest144430-21needreboot20:25:190
116792991715,1cyclictest0-21swapper/022:19:000
116792991715,1cyclictest0-21swapper/020:09:020
116792991710,5cyclictest206657-21awk23:25:110
11679299160,14cyclictest230718-21cron00:34:590
11680399150,3cyclictest144117-21cstates20:25:143
11679299155,8cyclictest228399-21python300:25:220
11679299155,8cyclictest228399-21python300:25:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional