You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 04:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Jun 28, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
285778522110,6sleep342-21ksoftirqd/319:05:163
295310821480,4sleep33056-21PK-Backend23:35:193
292347121090,3sleep10-21swapper/122:10:201
289153721070,3sleep13056-21PK-Backend20:40:131
292010621060,8sleep234-21ksoftirqd/222:00:232
1390991030,83rtkit-daemon0-21swapper/019:09:170
28842532680,4sleep13056-21PK-Backend20:20:131
28842532680,4sleep13056-21PK-Backend20:20:121
285866226342,14sleep20-21swapper/219:07:202
285864626343,14sleep10-21swapper/119:07:071
28812342510,3sleep20-21swapper/220:10:162
28743042450,3sleep30-21swapper/319:50:153
29438322360,5sleep00-21swapper/023:10:130
2859028993211,5cyclictest2968194-21wc00:20:002
2859028993011,5cyclictest2891559-21forks20:40:152
2859028993010,5cyclictest2949228-21cat23:25:172
29728572290,4sleep00-21swapper/000:30:210
285902899299,5cyclictest2874817-21cat19:50:202
29548322280,4chrt0-21swapper/123:40:181
285902899287,5cyclictest2945533-21ls23:15:122
285902899281,10cyclictest2969135-21python300:20:222
285902899278,5cyclictest2965120-21expr00:10:152
285902899272,7cyclictest2884231-21apt20:20:122
285902899272,7cyclictest2884231-21apt20:20:122
2859028992712,2cyclictest2974656-21python300:35:202
2859028992711,3cyclictest2872541-21if_enp2s019:45:152
285903399262,8cyclictest2969759-40unattended-upgr00:21:323
285903399261,22cyclictest2896128-21cron20:55:003
2859019992616,8cyclictest2867475-21memory19:30:150
2859028992510,2cyclictest2868606-21/usr/sbin/munin19:35:222
285902899250,18cyclictest0-21swapper/222:23:572
285902899250,18cyclictest0-21swapper/222:23:562
2859024992520,3cyclictest2889159-21cron20:35:001
285902899249,3cyclictest978-21snmpd23:46:392
285902899242,5cyclictest2936524-21apt22:50:142
2859028992416,7cyclictest0-21swapper/220:54:472
285901999240,22cyclictest2869591-21sed19:35:220
29590832230,4sleep00-21swapper/023:55:010
2859033992320,2cyclictest3056-21PK-Backend20:46:543
285902899230,22cyclictest0-21swapper/223:07:162
139099230,3rtkit-daemon1389-21rtkit-daemon21:47:002
2859033992220,1cyclictest3056-21PK-Backend21:25:543
2859033992219,2cyclictest3056-21PK-Backend21:05:063
285903399220,21cyclictest3056-21PK-Backend23:56:553
285903399220,21cyclictest3056-21PK-Backend21:18:513
285903399220,20cyclictest3056-21PK-Backend23:50:263
285903399220,20cyclictest3056-21PK-Backend23:49:523
285903399220,20cyclictest3056-21PK-Backend22:01:373
285903399220,20cyclictest3056-21PK-Backend21:12:153
285903399220,20cyclictest3056-21PK-Backend19:56:593
2859028992220,1cyclictest0-21swapper/220:08:572
2859028992210,3cyclictest141rcu_preempt00:00:002
2859024992214,6cyclictest883-21dbus-daemon00:15:001
2859024992213,7cyclictest2911092-21memory21:35:171
285902499220,21cyclictest2877803-21kernelversion20:00:151
285902499220,20cyclictest3056-21PK-Backend20:39:561
2859019992220,1cyclictest0-21swapper/022:28:260
2859019992220,1cyclictest0-21swapper/021:06:130
2859019992215,5cyclictest13-21ksoftirqd/021:25:130
29330462210,11sleep126-21ksoftirqd/122:40:111
28952312210,3sleep00-21swapper/020:50:160
2859033992119,1cyclictest3056-21PK-Backend22:26:443
2859033992119,1cyclictest0-21swapper/323:05:173
2859033992119,1cyclictest0-21swapper/322:10:223
2859033992119,1cyclictest0-21swapper/320:37:003
2859033992119,1cyclictest0-21swapper/319:13:463
2859033992118,2cyclictest3056-21PK-Backend23:14:383
2859033992118,2cyclictest3056-21PK-Backend19:34:493
2859033992117,2cyclictest3056-21PK-Backend21:00:053
285903399211,5cyclictest2938997-21ntp_states22:55:203
2859033992114,5cyclictest2870243-21sed19:40:003
285903399210,20cyclictest3056-21PK-Backend21:59:063
285903399210,20cyclictest0-21swapper/322:52:263
285903399210,20cyclictest0-21swapper/321:35:483
285903399210,19cyclictest3056-21PK-Backend22:46:523
285903399210,19cyclictest3056-21PK-Backend22:41:193
285903399210,19cyclictest3056-21PK-Backend00:16:223
2859028992119,1cyclictest3056-21PK-Backend21:40:232
2859028992119,1cyclictest0-21swapper/223:38:122
2859028992119,1cyclictest0-21swapper/222:31:402
2859028992119,1cyclictest0-21swapper/222:14:302
2859028992119,1cyclictest0-21swapper/221:27:052
2859028992119,1cyclictest0-21swapper/219:25:452
2859028992119,1cyclictest0-21swapper/219:25:452
2859028992119,1cyclictest0-21swapper/200:05:092
285902899210,20cyclictest0-21swapper/220:58:322
285902899210,20cyclictest0-21swapper/200:03:062
285902899210,1cyclictest0-21swapper/219:11:392
2859024992119,1cyclictest3056-21PK-Backend19:47:451
2859024992116,3cyclictest2919553-21diskstats22:00:131
2859024992113,7cyclictest2932066-21ntp_states22:35:191
285902499210,20cyclictest0-21swapper/119:30:141
285902499210,19cyclictest3056-21PK-Backend00:23:091
2859019992119,1cyclictest0-21swapper/022:30:360
2859019992119,1cyclictest0-21swapper/022:07:310
2859019992119,1cyclictest0-21swapper/021:49:530
2859019992119,1cyclictest0-21swapper/020:12:120
2859019992119,1cyclictest0-21swapper/020:08:130
2859019992119,1cyclictest0-21swapper/000:27:300
2859019992119,1cyclictest0-21swapper/000:05:060
2859019992119,1cyclictest0-21swapper/000:02:080
285901999210,20cyclictest0-21swapper/021:12:280
139099210,18rtkit-daemon0-21swapper/323:05:003
139099210,17rtkit-daemon2859013-21cyclictest21:30:263
285903399209,8cyclictest2974542-21/usr/sbin/munin00:35:193
2859033992018,1cyclictest3056-21PK-Backend21:50:563
2859033992018,1cyclictest3056-21PK-Backend20:08:143
2859033992018,1cyclictest3056-21PK-Backend19:25:493
2859033992018,1cyclictest3056-21PK-Backend19:25:493
2859033992018,1cyclictest3056-21PK-Backend19:20:423
2859033992018,1cyclictest0-21swapper/319:41:373
285903399200,1cyclictest0-21swapper/320:44:243
285903399200,19cyclictest3056-21PK-Backend23:42:493
285903399200,19cyclictest3056-21PK-Backend22:33:343
285903399200,19cyclictest0-21swapper/320:25:523
2859028992018,1cyclictest0-21swapper/222:36:542
2859028992018,1cyclictest0-21swapper/222:07:542
2859028992018,1cyclictest0-21swapper/221:55:442
2859028992018,1cyclictest0-21swapper/221:36:172
2859028992018,1cyclictest0-21swapper/221:20:022
2859028992018,1cyclictest0-21swapper/220:46:562
2859028992018,1cyclictest0-21swapper/220:37:042
2859028992018,1cyclictest0-21swapper/200:27:192
2859028992013,5cyclictest2864862-21cut19:25:002
285902899200,1cyclictest0-21swapper/223:44:202
285902899200,19cyclictest0-21swapper/222:46:272
285902899200,19cyclictest0-21swapper/220:34:402
285902499209,9cyclictest2953884-21unin-run23:40:001
2859024992018,1cyclictest3056-21PK-Backend22:51:031
2859024992018,1cyclictest0-21swapper/122:28:121
2859024992018,1cyclictest0-21swapper/100:15:531
2859024992013,6cyclictest2904195-21awk21:15:231
2859024992012,6cyclictest2943904-21cat23:10:151
285902499200,1cyclictest3056-21PK-Backend00:33:581
285902499200,1cyclictest0-21swapper/123:19:021
285902499200,1cyclictest0-21swapper/100:00:181
285902499200,19cyclictest3056-21PK-Backend23:23:141
285902499200,19cyclictest3056-21PK-Backend21:23:161
285902499200,19cyclictest3056-21PK-Backend19:52:391
285902499200,19cyclictest0-21swapper/122:20:211
285902499200,19cyclictest0-21swapper/122:20:211
285902499200,19cyclictest0-21swapper/120:50:381
285902499200,19cyclictest0-21swapper/119:36:291
285902499200,19cyclictest0-21swapper/119:28:451
285902499200,19cyclictest0-21swapper/119:28:451
285902499200,19cyclictest0-21swapper/119:21:161
2859019992018,1cyclictest0-21swapper/021:37:290
2859019992018,1cyclictest0-21swapper/019:53:130
2859019992018,1cyclictest0-21swapper/019:49:480
2859019992017,2cyclictest0-21swapper/021:55:140
2859019992013,5cyclictest2927594-21perf22:25:000
2859019992013,5cyclictest2927594-21perf22:25:000
285901999200,19cyclictest0-21swapper/022:51:450
139099200,17rtkit-daemon1997-21gvfs-afc-volume21:18:422
139099200,14rtkit-daemon0-21swapper/121:04:221
139099200,13rtkit-daemon0-21swapper/120:06:461
139099200,13rtkit-daemon0-21swapper/020:57:540
2859033991917,1cyclictest3056-21PK-Backend21:46:503
2859033991917,1cyclictest0-21swapper/322:17:193
2859033991917,1cyclictest0-21swapper/320:30:593
2859033991916,2cyclictest3056-21PK-Backend20:57:363
2859033991916,1cyclictest3056-21PK-Backend20:00:383
2859033991912,5cyclictest2970805-21awk00:25:173
285903399191,16cyclictest2946650-21apt23:20:003
285903399190,1cyclictest3056-21PK-Backend21:44:193
285903399190,1cyclictest3056-21PK-Backend00:04:193
285903399190,1cyclictest0-21swapper/321:22:433
285903399190,18cyclictest3056-21PK-Backend22:35:433
2859028991917,1cyclictest0-21swapper/222:56:002
2859028991917,1cyclictest0-21swapper/221:50:442
2859028991917,1cyclictest0-21swapper/221:06:372
2859028991917,1cyclictest0-21swapper/221:01:572
2859028991917,1cyclictest0-21swapper/200:34:362
285902899190,1cyclictest0-21swapper/223:21:222
285902899190,1cyclictest0-21swapper/222:18:352
285902899190,1cyclictest0-21swapper/219:59:512
285902499199,8cyclictest2910275-21unin-run21:35:001
2859024991917,1cyclictest0-21swapper/121:05:591
2859024991917,1cyclictest0-21swapper/120:11:311
2859024991915,3cyclictest2963666-21sensors_temp00:05:211
2859024991912,5cyclictest2915879-21expr21:50:131
2859024991911,6cyclictest2917438-21awk21:55:111
2859024991911,6cyclictest2897457-21awk20:55:221
285902499190,1cyclictest3056-21PK-Backend19:41:501
285902499190,1cyclictest0-21swapper/123:46:121
285902499190,1cyclictest0-21swapper/122:47:591
285902499190,1cyclictest0-21swapper/121:46:071
285902499190,1cyclictest0-21swapper/120:19:081
285902499190,1cyclictest0-21swapper/119:57:311
285902499190,18cyclictest0-21swapper/123:30:101
285902499190,18cyclictest0-21swapper/121:26:471
285902499190,18cyclictest0-21swapper/121:14:041
285902499190,18cyclictest0-21swapper/120:46:321
2859019991917,1cyclictest3056-21PK-Backend22:55:300
2859019991917,1cyclictest0-21swapper/023:47:210
2859019991917,1cyclictest0-21swapper/022:46:450
2859019991917,1cyclictest0-21swapper/021:53:590
2859019991917,1cyclictest0-21swapper/019:26:580
2859019991917,1cyclictest0-21swapper/019:26:570
2859019991914,3cyclictest2960291-21timerandwakeup23:55:220
2859019991911,6cyclictest436-20systemd-journal19:24:590
285901999191,16cyclictest2923753-21nm-dispatcher22:10:370
2859019991910,7cyclictest883-21dbus-daemon23:30:000
285901999190,3cyclictest2951859-21munin-run23:35:000
285901999190,1cyclictest0-21swapper/020:47:000
285901999190,18cyclictest0-21swapper/023:16:390
285901999190,18cyclictest0-21swapper/022:35:160
285901999190,18cyclictest0-21swapper/020:35:450
285901999190,18cyclictest0-21swapper/019:57:320
285901999190,18cyclictest0-21swapper/000:10:260
285903399186,8cyclictest2947790-21python323:20:213
2859033991816,1cyclictest3056-21PK-Backend23:32:423
2859033991816,1cyclictest3056-21PK-Backend22:06:373
2859033991816,1cyclictest3056-21PK-Backend20:15:513
2859033991816,1cyclictest3056-21PK-Backend00:13:453
2859033991816,1cyclictest0-21swapper/322:22:393
2859033991816,1cyclictest0-21swapper/322:22:393
2859033991816,1cyclictest0-21swapper/300:32:033
285903399181,15cyclictest2872457-21df19:45:123
285903399180,16cyclictest2861809-21diskstats19:15:143
285902899183,13cyclictest2944934-21idleruntime-cro23:15:002
2859028991816,1cyclictest0-21swapper/221:30:172
2859028991816,1cyclictest0-21swapper/221:13:262
2859024991816,1cyclictest3056-21PK-Backend00:36:351
2859024991816,1cyclictest0-21swapper/121:40:261
285901999189,7cyclictest2953460-21users23:35:230
2859019991816,1cyclictest0-21swapper/022:43:200
2859019991816,1cyclictest0-21swapper/022:03:250
2859019991816,1cyclictest0-21swapper/021:32:220
2859019991816,1cyclictest0-21swapper/021:17:290
2859019991816,1cyclictest0-21swapper/020:03:200
2859019991816,1cyclictest0-21swapper/019:40:080
2859019991813,3cyclictest2889172-21sh20:35:000
2859019991812,5cyclictest2954409-21ls23:40:130
2859019991812,5cyclictest2899209-21sshd21:00:240
2859019991811,5cyclictest2967164-21cat00:15:190
2859019991810,6cyclictest436-20systemd-journal21:45:000
285903399176,10cyclictest3056-21PK-Backend00:06:223
285903399170,3cyclictest2884627-21hddtemp_smartct20:20:163
285903399170,3cyclictest2884627-21hddtemp_smartct20:20:163
285902899179,6cyclictest2861059-21df_inode19:15:032
2859028991715,1cyclictest0-21swapper/220:29:322
2859028991713,3cyclictest2878806-21latency_hist20:05:012
2859028991710,5cyclictest2933218-21tr22:40:122
2859024991714,2cyclictest3056-21PK-Backend19:12:291
2859024991714,2cyclictest0-21swapper/122:05:151
2859024991712,3cyclictest2958591-21timerwakeupswit23:50:221
2859024991710,5cyclictest2942521-21seq23:05:201
285901999179,5cyclictest2924545-21grep22:15:110
2859019991715,1cyclictest0-21swapper/023:23:220
2859019991715,1cyclictest0-21swapper/019:16:360
139099170,14rtkit-daemon0-21swapper/223:00:412
2859033991614,1cyclictest3056-21PK-Backend20:10:033
285903399161,11cyclictest2948660-21apt23:25:123
285902899169,5cyclictest2871753-21seq19:44:152
2859028991614,1cyclictest0-21swapper/223:51:042
2859028991611,3cyclictest1-21systemd19:35:002
2859024991614,1cyclictest3056-21PK-Backend23:00:581
2859024991614,1cyclictest3056-21PK-Backend00:29:521
2859024991611,3cyclictest2929563-21/usr/sbin/munin22:30:131
2859019991614,1cyclictest0-21swapper/000:36:220
2859019991614,1cyclictest0-21swapper/000:23:040
2859019991612,3cyclictest2886604-21needreboot20:25:170
2859019991612,3cyclictest2882468-21/usr/sbin/munin20:15:190
285902899159,4cyclictest2950944-21cat23:30:162
2859028991513,1cyclictest0-21swapper/220:15:192
285902499158,5cyclictest436-20systemd-journal23:29:561
2859024991510,3cyclictest2959367-21cpu23:55:121
2859024991510,3cyclictest2925390-21timerwakeupswit22:15:201
285901999159,5cyclictest2943176-21NetworkManager23:10:000
285901999159,4cyclictest2860680-21dump-pmu-power19:15:000
285901999158,5cyclictest2905561-21cut21:20:180
285902499148,4cyclictest2156-21gnome-shell22:55:091
285901999142,11cyclictest2891658-21kernelversion20:40:170
285901999140,1cyclictest0-21swapper/020:23:150
285901999140,1cyclictest0-21swapper/020:23:140
285902899131,10cyclictest2927906-21/usr/sbin/munin22:25:132
285902499139,3cyclictest3056-21PK-Backend20:28:341
2859019991311,1cyclictest0-21swapper/023:00:510
139099130,4rtkit-daemon3056-21PK-Backend19:16:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional