You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 12:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Sat Apr 11, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
73716121320,3sleep20-21swapper/200:30:152
6724372980,4sleep13056-21PK-Backend21:25:201
6966782890,2sleep30-21swapper/322:35:163
62143027939,33sleep00-21swapper/019:05:130
62275627738,13sleep10-21swapper/119:08:311
62250227444,21sleep30-21swapper/319:05:293
62260426242,14sleep20-21swapper/219:06:352
7055002500,4sleep13056-21PK-Backend23:00:161
7283272440,5sleep13056-21PK-Backend00:05:201
139099440,5rtkit-daemon0-21swapper/221:00:372
62302899320,23cyclictest6880741missed_timers22:10:163
62302899300,27cyclictest436-20systemd-journal00:10:003
7178142290,4sleep13056-21PK-Backend23:35:161
62302899281,25cyclictest2173-21JS10:27:483
62302899281,22cyclictest678519-21apt21:45:123
623028992811,15cyclictest651094-21sed20:25:163
62302899273,15cyclictest0-21swapper/300:17:413
62302899272,21cyclictest625437-21apt19:15:123
62302899271,24cyclictest667477-21chrt21:10:393
62302899271,24cyclictest667477-21chrt21:10:393
62302899271,24cyclictest624680-21munin-run19:14:593
62302899270,25cyclictest655279-21chrt20:38:073
139099270,18rtkit-daemon34-21ksoftirqd/220:35:212
62302899262,22cyclictest715652-21cstates23:30:143
62302899262,20cyclictest2156-21gnome-shell21:01:093
62302899261,22cyclictest721587-21awk23:45:213
62302899261,21cyclictest672439-21python321:25:213
62302899261,21cyclictest638105-21apt19:50:113
623028992612,12cyclictest734915-21cpuspeed_turbos00:25:133
623028992612,11cyclictest681720-21latency_hist21:55:013
62302899261,20cyclictest632645-21apt19:35:123
623028992610,14cyclictest685618-21date22:05:003
623028992610,14cyclictest682875-21cat21:55:183
623028992610,14cyclictest669768-21wc21:20:003
623028992610,14cyclictest657673-21expr20:45:123
623028992610,14cyclictest436-20systemd-journal20:58:383
62302899260,2cyclictest0-21swapper/321:34:563
62302899260,24cyclictest630387-21latency_hist19:30:013
62302899259,14cyclictest695923-21wc22:35:003
62302899258,15cyclictest676073-21cat21:35:203
62302899256,17cyclictest700668-21seq22:45:233
62302899256,17cyclictest1-21systemd23:05:003
62302899254,20cyclictest42-21ksoftirqd/322:05:173
62302899252,21cyclictest729839-21expr00:10:173
62302899252,20cyclictest2156-21gnome-shell19:45:093
62302899252,20cyclictest0-21swapper/323:10:113
62302899252,20cyclictest0-21swapper/319:20:093
62302899252,20cyclictest0-21swapper/319:20:093
62302899252,19cyclictest702786-21apt-get22:55:003
62302899252,19cyclictest0-21swapper/323:40:103
62302899252,18cyclictest689180-21apt22:15:133
62302899252,18cyclictest0-21swapper/319:55:113
62302899251,22cyclictest632110-21latency_hist19:35:013
62302899251,21cyclictest711099-21python323:15:193
62302899251,20cyclictest659403-21apt-get20:50:043
623028992510,13cyclictest704910-21date23:00:023
623028992510,13cyclictest634095-21cron19:44:553
62302899249,13cyclictest642038-21cstates20:00:133
62302899247,16cyclictest42-21ksoftirqd/321:22:063
62302899243,18cyclictest664759-21cpuspeed_turbos21:05:133
62302899243,17cyclictest2174-21JS10:27:483
62302899241,20cyclictest736497-21/usr/sbin/munin00:30:123
62302899241,20cyclictest707629-21python323:05:193
62302899240,22cyclictest653424-21anacron20:31:353
62302899240,12cyclictest0-21swapper/300:35:213
623019992415,5cyclictest736262-21munin-plugin-st00:30:001
623014992415,7cyclictest637567-21apt-get19:50:000
139099240,18rtkit-daemon34-21ksoftirqd/222:23:262
139099240,15rtkit-daemon34-21ksoftirqd/220:22:112
7333172230,9sleep126-21ksoftirqd/100:20:111
62302899237,14cyclictest691336-21cat22:20:153
62302899231,20cyclictest715177-21perf23:30:003
62302899231,19cyclictest717180-21apt23:35:113
62302899231,19cyclictest692824-21grep22:25:143
62302899231,19cyclictest657193-21awk20:45:003
62302899231,18cyclictest724109-21apt23:55:103
62302899230,21cyclictest698331-21kernelversion22:40:153
623019992320,2cyclictest3056-21PK-Backend21:37:171
623019992313,8cyclictest632112-21cron19:35:001
623014992316,5cyclictest13-21ksoftirqd/020:10:110
139099230,16rtkit-daemon34-21ksoftirqd/221:09:012
62302899223,17cyclictest726180-21cat00:00:153
62302899221,19cyclictest734061-21wc00:20:213
62302899221,19cyclictest643771-21cstates20:05:123
62302899220,20cyclictest722823-21/usr/sbin/munin23:50:153
623024992220,1cyclictest0-21swapper/222:39:232
623024992220,1cyclictest0-21swapper/220:18:432
623024992219,2cyclictest3056-21PK-Backend20:14:182
623024992216,5cyclictest625840-21idleruntime19:15:162
623024992212,8cyclictest656158-21cat20:40:162
62302499220,21cyclictest978-21snmpd23:19:382
62302499220,1cyclictest0-21swapper/223:06:542
623019992220,1cyclictest3056-21PK-Backend19:20:101
623019992220,1cyclictest3056-21PK-Backend19:20:091
623019992220,1cyclictest0-21swapper/120:36:341
623019992219,2cyclictest3056-21PK-Backend22:24:011
623019992219,2cyclictest3056-21PK-Backend22:13:471
623019992219,2cyclictest3056-21PK-Backend21:02:501
62301999220,20cyclictest3056-21PK-Backend21:42:171
62301999220,20cyclictest3056-21PK-Backend20:40:331
62301999220,20cyclictest3056-21PK-Backend20:26:271
62301999220,20cyclictest3056-21PK-Backend19:50:101
62301999220,20cyclictest3056-21PK-Backend00:35:491
623014992220,1cyclictest0-21swapper/021:41:040
623014992220,1cyclictest0-21swapper/021:33:060
623014992220,1cyclictest0-21swapper/021:01:030
62301499220,21cyclictest0-21swapper/021:48:130
62302899211,18cyclictest711696-21cron23:20:013
62302899211,18cyclictest648513-21awk20:20:003
62302899210,4cyclictest2306-21gsd-housekeepin20:12:563
623024992119,1cyclictest0-21swapper/223:56:152
623024992119,1cyclictest0-21swapper/223:42:502
623024992119,1cyclictest0-21swapper/223:36:082
623024992119,1cyclictest0-21swapper/223:23:302
623024992119,1cyclictest0-21swapper/222:25:032
623024992119,1cyclictest0-21swapper/222:18:002
623024992119,1cyclictest0-21swapper/222:03:062
623024992119,1cyclictest0-21swapper/220:51:282
623024992119,1cyclictest0-21swapper/220:28:572
623024992119,1cyclictest0-21swapper/219:29:492
623024992119,1cyclictest0-21swapper/219:21:312
623024992119,1cyclictest0-21swapper/219:21:312
623024992119,1cyclictest0-21swapper/200:16:572
623024992119,1cyclictest0-21swapper/200:03:462
62302499210,19cyclictest3056-21PK-Backend20:30:322
623019992119,1cyclictest3056-21PK-Backend23:21:071
623019992119,1cyclictest3056-21PK-Backend21:31:431
623019992119,1cyclictest3056-21PK-Backend21:24:001
623019992119,1cyclictest3056-21PK-Backend21:17:521
623019992119,1cyclictest3056-21PK-Backend20:46:071
623019992119,1cyclictest3056-21PK-Backend00:15:011
623019992117,3cyclictest26-21ksoftirqd/120:10:111
62301999210,20cyclictest0-21swapper/122:00:091
62301999210,19cyclictest3056-21PK-Backend23:11:131
62301999210,19cyclictest3056-21PK-Backend22:27:261
62301999210,19cyclictest3056-21PK-Backend22:17:121
62301999210,19cyclictest3056-21PK-Backend19:35:431
62301999210,19cyclictest3056-21PK-Backend19:18:211
623014992119,1cyclictest0-21swapper/023:48:460
623014992119,1cyclictest0-21swapper/023:35:280
623014992119,1cyclictest0-21swapper/021:58:490
623014992119,1cyclictest0-21swapper/021:29:220
623014992119,1cyclictest0-21swapper/020:59:260
623014992119,1cyclictest0-21swapper/020:48:310
623014992119,1cyclictest0-21swapper/020:19:320
623014992119,1cyclictest0-21swapper/019:59:310
623014992119,1cyclictest0-21swapper/019:12:400
623014992119,1cyclictest0-21swapper/000:05:290
623014992118,2cyclictest3056-21PK-Backend00:25:230
623014992118,2cyclictest0-21swapper/022:06:120
623014992117,3cyclictest0-21swapper/023:55:010
62301499210,20cyclictest3056-21PK-Backend23:53:120
62301499210,20cyclictest0-21swapper/022:39:580
62301499210,20cyclictest0-21swapper/020:35:270
62301499210,20cyclictest0-21swapper/020:33:040
139099210,14rtkit-daemon0-21swapper/200:06:312
7196592200,4sleep13056-21PK-Backend23:40:181
62302499205,14cyclictest715165-21sh23:30:002
623024992018,1cyclictest0-21swapper/223:50:342
623024992018,1cyclictest0-21swapper/223:32:162
623024992018,1cyclictest0-21swapper/223:01:272
623024992018,1cyclictest0-21swapper/221:43:132
623024992018,1cyclictest0-21swapper/221:15:072
623024992018,1cyclictest0-21swapper/220:57:162
623024992018,1cyclictest0-21swapper/219:58:492
623024992018,1cyclictest0-21swapper/219:43:282
623024992018,1cyclictest0-21swapper/200:14:062
62302499200,1cyclictest0-21swapper/221:21:152
62302499200,1cyclictest0-21swapper/200:21:372
62302499200,19cyclictest0-21swapper/222:56:472
623019992018,1cyclictest3056-21PK-Backend22:51:261
623019992018,1cyclictest3056-21PK-Backend21:06:351
623019992018,1cyclictest3056-21PK-Backend00:03:041
623019992018,1cyclictest0-21swapper/121:10:201
623019992018,1cyclictest0-21swapper/121:10:201
623019992017,2cyclictest3056-21PK-Backend22:06:101
623019992017,2cyclictest3056-21PK-Backend20:03:211
623019992017,2cyclictest3056-21PK-Backend19:57:341
623019992013,5cyclictest644274-21ntpq20:05:171
62301999200,19cyclictest3056-21PK-Backend23:17:291
62301999200,19cyclictest3056-21PK-Backend21:54:551
62301999200,19cyclictest3056-21PK-Backend21:45:431
62301999200,19cyclictest3056-21PK-Backend19:41:261
62301999200,18cyclictest3056-21PK-Backend23:45:271
62301999200,18cyclictest3056-21PK-Backend20:22:151
623014992019,1cyclictest0-21swapper/019:42:270
623014992018,1cyclictest0-21swapper/019:31:460
623014992018,1cyclictest0-21swapper/019:28:090
623014992018,1cyclictest0-21swapper/000:03:400
623014992013,5cyclictest644074-21wc20:05:150
62301499200,1cyclictest0-21swapper/020:24:320
62301499200,19cyclictest0-21swapper/023:28:110
62301499200,19cyclictest0-21swapper/023:00:400
139099200,14rtkit-daemon0-21swapper/200:28:282
623024991917,1cyclictest0-21swapper/222:12:462
623024991917,1cyclictest0-21swapper/221:26:022
623024991917,1cyclictest0-21swapper/219:45:312
623024991917,1cyclictest0-21swapper/219:37:342
623024991913,5cyclictest683564-21seq21:58:192
623024991913,5cyclictest631380-21ntpq19:30:162
623024991912,5cyclictest701456-21expr22:50:112
623024991912,5cyclictest1095-21runrttasks20:00:112
623024991911,6cyclictest658901-21cron20:50:002
623024991910,7cyclictest739331-21users00:35:222
62302499190,1cyclictest3056-21PK-Backend19:10:162
62302499190,1cyclictest0-21swapper/223:14:112
62302499190,18cyclictest0-21swapper/222:08:212
623019991917,1cyclictest3056-21PK-Backend22:56:531
623019991917,1cyclictest3056-21PK-Backend22:45:581
623019991917,1cyclictest3056-21PK-Backend21:56:441
623019991917,1cyclictest3056-21PK-Backend00:34:001
623019991917,1cyclictest3056-21PK-Backend00:11:221
62301999190,1cyclictest3056-21PK-Backend23:58:251
62301999190,1cyclictest3056-21PK-Backend23:50:061
62301999190,1cyclictest3056-21PK-Backend23:25:461
62301999190,1cyclictest3056-21PK-Backend23:07:351
62301999190,1cyclictest3056-21PK-Backend22:39:431
62301999190,1cyclictest3056-21PK-Backend19:26:241
62301999190,18cyclictest0-21swapper/119:45:171
623014991917,1cyclictest0-21swapper/023:05:540
623014991917,1cyclictest0-21swapper/022:45:120
623014991917,1cyclictest0-21swapper/022:34:100
623014991917,1cyclictest0-21swapper/020:26:560
623014991917,1cyclictest0-21swapper/020:03:300
623014991917,1cyclictest0-21swapper/000:22:460
623014991912,5cyclictest738460-21cat00:35:130
62301499190,1cyclictest0-21swapper/022:16:460
62301499190,1cyclictest0-21swapper/021:06:430
139099190,13rtkit-daemon0-21swapper/223:45:522
623024991816,1cyclictest0-21swapper/222:46:402
623024991816,1cyclictest0-21swapper/222:31:182
623024991816,1cyclictest0-21swapper/221:13:332
623024991816,1cyclictest0-21swapper/221:13:322
623024991810,6cyclictest698815-21sed22:40:202
623024991810,6cyclictest436-20systemd-journal21:55:002
623019991816,1cyclictest3056-21PK-Backend20:19:381
623019991816,1cyclictest0-21swapper/120:34:111
623019991811,5cyclictest698548-21awk22:40:181
623014991816,1cyclictest0-21swapper/023:23:450
623014991816,1cyclictest0-21swapper/023:18:310
623014991816,1cyclictest0-21swapper/023:14:180
623014991816,1cyclictest0-21swapper/022:42:350
623014991816,1cyclictest0-21swapper/022:23:230
623014991816,1cyclictest0-21swapper/020:42:230
623014991815,2cyclictest0-21swapper/022:57:150
623014991812,4cyclictest668271-21cpuspeed_turbos21:15:120
7317532170,3sleep0731794-21ntp_states00:15:180
623024991715,1cyclictest0-21swapper/219:54:362
623024991712,3cyclictest678566-21cpu21:45:122
62302499171,12cyclictest675055-21apt21:35:122
623019991715,1cyclictest3056-21PK-Backend23:33:171
623019991715,1cyclictest3056-21PK-Backend22:31:521
623019991711,4cyclictest623341-21tr19:10:111
623014991715,1cyclictest0-21swapper/023:31:010
623014991715,1cyclictest0-21swapper/019:23:420
623014991715,1cyclictest0-21swapper/019:23:410
623014991715,1cyclictest0-21swapper/000:10:430
623014991714,2cyclictest685430-21cron22:04:590
623014991711,5cyclictest680746-21sed21:50:160
62301499170,16cyclictest0-21swapper/021:23:260
62301499169,6cyclictest688504-21sed22:10:200
623014991614,1cyclictest0-21swapper/023:42:310
623014991614,1cyclictest0-21swapper/022:50:270
623014991613,2cyclictest692813-21tr22:25:130
623014991610,4cyclictest737008-21cut00:30:140
62301499160,1cyclictest0-21swapper/019:15:510
623024991513,1cyclictest0-21swapper/221:30:012
62302499150,13cyclictest643895-21cat20:05:132
62301499159,4cyclictest660023-21grep20:50:150
623014991513,1cyclictest3056-21PK-Backend21:11:500
623014991513,1cyclictest3056-21PK-Backend21:11:500
623014991511,3cyclictest633040-21hddtemp_smartct19:35:150
623019991410,3cyclictest3056-21PK-Backend20:53:341
62301499147,6cyclictest638738-21cat19:50:180
62301499140,5cyclictest3056-21PK-Backend21:35:190
62301999111,8cyclictest662629-21cron21:00:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional