You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 03:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Jul 28, 2026 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
319096821060,4sleep20-21swapper/222:45:002
139099730,61rtkit-daemon0-21swapper/119:09:531
311460626640,20sleep20-21swapper/219:05:462
311492526446,13sleep30-21swapper/319:09:533
311490026446,13sleep00-21swapper/019:09:340
31321392370,1chrt3131354-21/usr/sbin/munin19:55:211
311507699370,23cyclictest3129256-21taskset19:48:391
311507699370,16cyclictest0-21swapper/120:10:011
311507699360,22cyclictest0-21swapper/122:25:171
311507699350,22cyclictest0-21swapper/120:48:131
311507699350,20cyclictest3199486-21seq23:06:521
311507699340,21cyclictest3213434-21seq23:47:541
311507699340,19cyclictest3127597-21sshd19:44:371
311507699330,17cyclictest3218721-21taskset00:03:501
311507699330,17cyclictest0-21swapper/119:15:261
311507699320,18cyclictest3229177-21taskset00:34:341
311507699320,17cyclictest3215052-21taskset23:51:261
311507699320,17cyclictest3208272-21seq23:33:221
311507699311,27cyclictest3168285-21cron21:40:001
311507699311,18cyclictest3122689-21taskset19:30:111
311507699310,17cyclictest3202897-21seq23:16:101
311507699310,17cyclictest3173312-21runrttasks21:52:491
311507699310,17cyclictest3120469-21taskset19:22:161
311507699310,17cyclictest0-21swapper/123:57:111
311507699310,16cyclictest3159600-21taskset21:14:481
311507699310,14cyclictest0-21swapper/120:40:121
311507699300,17cyclictest3162819-21seq21:21:381
311507699300,16cyclictest3161231-21seq21:18:291
311507699300,16cyclictest3141872-21seq20:23:031
311507699300,16cyclictest3139697-21sensors20:15:201
311507699300,16cyclictest0-21swapper/121:57:391
311507699300,16cyclictest0-21swapper/100:36:061
311507699300,15cyclictest3145416-21taskset20:34:021
311507699290,18cyclictest3122405-21chrt19:29:431
311507699290,16cyclictest3204806-21seq23:23:241
311507699290,16cyclictest3179839-21/usr/sbin/munin22:10:211
311507699290,16cyclictest0-21swapper/122:16:191
311507699290,15cyclictest3171675-21seq21:49:041
311507699281,16cyclictest0-21swapper/119:37:001
311507699280,16cyclictest0-21swapper/121:42:391
311507699280,16cyclictest0-21swapper/119:51:471
311507699280,15cyclictest0-21swapper/122:22:571
311507699280,15cyclictest0-21swapper/120:51:181
311507699270,15cyclictest3208691-21/usr/sbin/munin23:35:131
311507699270,15cyclictest0-21swapper/123:43:011
311507699270,14cyclictest3116678-21runrttasks19:14:261
311507699260,16cyclictest3192428-21taskset22:48:321
311507699260,15cyclictest0-21swapper/122:08:371
311507699260,14cyclictest0-21swapper/121:26:031
3115072992621,4cyclictest13-21ksoftirqd/019:24:160
3115072992614,10cyclictest3210177-21unin-run23:40:000
31305892250,3sleep33056-21PK-Backend19:50:223
31179052250,3sleep20-21swapper/219:15:442
311507699258,15cyclictest3142517-21cat20:25:141
311507699250,15cyclictest3222132-21taskset00:13:091
311507699250,14cyclictest0-21swapper/123:27:571
311507699250,14cyclictest0-21swapper/123:12:451
311507699250,14cyclictest0-21swapper/100:05:101
311507299259,4cyclictest13-21ksoftirqd/019:43:250
3115081992414,7cyclictest42-21ksoftirqd/321:30:203
3115076992419,4cyclictest0-21swapper/122:42:411
311507699240,14cyclictest1095-21runrttasks00:17:451
3115072992416,6cyclictest3201496-21idleruntime-cro23:15:000
31658002230,4sleep20-21swapper/221:30:182
311508199230,3cyclictest3188002-21df_inode22:35:133
311508199230,21cyclictest3056-21PK-Backend20:33:543
311508199230,21cyclictest3056-21PK-Backend00:02:483
3115080992321,1cyclictest0-21swapper/200:30:302
3115080992315,6cyclictest3141440-21python320:20:212
311508099230,3cyclictest3200762-21ntp_states23:10:202
311507299237,4cyclictest13-21ksoftirqd/000:24:440
3115072992311,3cyclictest141rcu_preempt20:06:320
31838852220,3chrt3056-21PK-Backend22:25:003
3115081992220,1cyclictest0-21swapper/321:02:133
3115081992220,1cyclictest0-21swapper/300:26:483
3115081992219,2cyclictest3056-21PK-Backend19:26:353
311508199220,21cyclictest0-21swapper/323:02:113
311508199220,20cyclictest3056-21PK-Backend22:05:403
311507699228,12cyclictest3194284-21cron22:55:001
311507699222,15cyclictest3145777-21apt20:35:121
3115076992217,4cyclictest0-21swapper/121:30:081
311507299229,4cyclictest13-21ksoftirqd/022:47:360
311507299229,3cyclictest13-21ksoftirqd/020:54:030
311507299227,4cyclictest13-21ksoftirqd/023:00:390
311507299226,4cyclictest13-21ksoftirqd/000:17:380
3115072992220,1cyclictest3056-21PK-Backend22:56:520
3115072992220,1cyclictest0-21swapper/022:01:150
3115072992220,1cyclictest0-21swapper/021:56:220
3115072992220,1cyclictest0-21swapper/020:16:380
3115072992219,2cyclictest0-21swapper/023:06:580
3115072992214,6cyclictest3214462-21memory23:50:180
3115072992214,6cyclictest3150106-21python320:45:220
311507299220,21cyclictest0-21swapper/022:15:350
311507299220,20cyclictest3056-21PK-Backend22:33:330
3115081992119,1cyclictest3056-21PK-Backend21:15:113
3115081992119,1cyclictest3056-21PK-Backend20:48:143
3115081992119,1cyclictest3056-21PK-Backend20:23:263
3115081992119,1cyclictest0-21swapper/322:51:303
3115081992119,1cyclictest0-21swapper/300:09:233
3115081992118,2cyclictest3056-21PK-Backend20:11:163
3115081992118,2cyclictest0-21swapper/322:25:213
311508199210,20cyclictest3056-21PK-Backend23:17:123
311508199210,20cyclictest0-21swapper/322:42:583
311508199210,19cyclictest3056-21PK-Backend23:50:243
311508199210,19cyclictest3056-21PK-Backend21:51:143
3115080992119,1cyclictest0-21swapper/223:50:292
3115080992116,3cyclictest34-21ksoftirqd/219:40:192
3115080992115,4cyclictest2156-21gnome-shell20:57:092
311508099210,20cyclictest0-21swapper/223:01:082
311508099210,20cyclictest0-21swapper/221:35:232
311508099210,20cyclictest0-21swapper/219:24:162
311507699217,12cyclictest0-21swapper/122:59:221
311507699215,14cyclictest3137089-21find20:10:121
311507699212,17cyclictest3186447-21sort22:30:181
311507699211,18cyclictest31564062sleep121:05:111
311507699211,17cyclictest3225167-21python300:20:211
311507699211,17cyclictest3175631-21cpuspeed_turbos22:00:121
311507699211,17cyclictest3134581-21perf20:05:001
311507299218,4cyclictest13-21ksoftirqd/000:13:330
3115072992119,1cyclictest3056-21PK-Backend22:42:390
3115072992119,1cyclictest0-21swapper/023:33:010
3115072992119,1cyclictest0-21swapper/020:43:010
3115072992119,1cyclictest0-21swapper/020:33:550
3115072992119,1cyclictest0-21swapper/020:26:590
3115072992119,1cyclictest0-21swapper/019:52:250
3115072992118,2cyclictest3056-21PK-Backend20:57:140
3115072992113,6cyclictest3116229-21expr19:10:240
3115072992110,2cyclictest141rcu_preempt00:30:490
311507299210,2cyclictest3230293-21ps00:35:200
311507299210,20cyclictest0-21swapper/023:41:260
311507299210,20cyclictest0-21swapper/020:35:510
311507299210,20cyclictest0-21swapper/019:46:160
311507299210,20cyclictest0-21swapper/000:00:380
311507299210,19cyclictest3056-21PK-Backend22:38:060
311507299210,19cyclictest3056-21PK-Backend00:27:570
139099210,14rtkit-daemon0-21swapper/019:15:130
31692932200,3sleep33056-21PK-Backend21:40:163
3115081992018,1cyclictest3056-21PK-Backend23:33:003
3115081992018,1cyclictest3056-21PK-Backend23:23:003
3115081992018,1cyclictest3056-21PK-Backend22:18:383
3115081992018,1cyclictest3056-21PK-Backend20:50:583
3115081992018,1cyclictest3056-21PK-Backend20:44:563
3115081992018,1cyclictest3056-21PK-Backend20:37:463
3115081992018,1cyclictest3056-21PK-Backend19:43:453
3115081992018,1cyclictest0-21swapper/323:55:243
3115081992018,1cyclictest0-21swapper/319:22:163
3115081992017,2cyclictest3056-21PK-Backend21:20:323
3115081992016,3cyclictest0-21swapper/322:45:213
311508199201,4cyclictest3227767-21apt00:30:133
3115081992013,5cyclictest436-20systemd-journal00:24:383
3115081992012,6cyclictest3143767-21cron20:30:003
311508199200,1cyclictest3056-21PK-Backend23:11:243
311508199200,1cyclictest3056-21PK-Backend22:14:533
311508199200,1cyclictest0-21swapper/321:10:233
311508199200,19cyclictest3056-21PK-Backend19:38:243
311508199200,19cyclictest3056-21PK-Backend19:12:223
311508199200,19cyclictest0-21swapper/323:06:243
311508199200,19cyclictest0-21swapper/322:56:513
3115080992018,1cyclictest0-21swapper/223:34:132
3115080992018,1cyclictest0-21swapper/222:54:112
3115080992018,1cyclictest0-21swapper/222:21:062
3115080992018,1cyclictest0-21swapper/222:03:212
3115080992018,1cyclictest0-21swapper/220:17:502
3115080992018,1cyclictest0-21swapper/220:01:412
3115080992018,1cyclictest0-21swapper/219:55:322
3115080992018,1cyclictest0-21swapper/219:28:012
3115080992018,1cyclictest0-21swapper/200:18:062
3115080992017,2cyclictest3056-21PK-Backend20:36:152
311508099201,6cyclictest141rcu_preempt19:38:112
3115080992013,5cyclictest3225408-21sshd00:20:372
3115080992013,5cyclictest3216319-21ntp_kernel_err23:55:202
3115080992012,2cyclictest34-21ksoftirqd/223:40:122
3115080992011,7cyclictest3163946-21memory21:25:182
311508099200,1cyclictest0-21swapper/222:47:142
311508099200,1cyclictest0-21swapper/200:13:542
311508099200,19cyclictest0-21swapper/223:38:252
311508099200,19cyclictest0-21swapper/223:26:362
311508099200,19cyclictest0-21swapper/223:23:592
311508099200,19cyclictest0-21swapper/222:38:432
311508099200,19cyclictest0-21swapper/222:25:392
311508099200,19cyclictest0-21swapper/221:18:542
311508099200,19cyclictest0-21swapper/220:54:542
311508099200,19cyclictest0-21swapper/220:11:142
311508099200,19cyclictest0-21swapper/219:49:242
311508099200,19cyclictest0-21swapper/219:34:442
311508099200,19cyclictest0-21swapper/200:39:572
311508099200,19cyclictest0-21swapper/200:07:192
3115076992018,1cyclictest0-21swapper/123:00:051
311507699201,17cyclictest3153957-21timerandwakeup20:55:241
311507699200,17cyclictest3155094-21/usr/sbin/munin21:00:171
3115072992018,1cyclictest0-21swapper/023:49:510
3115072992018,1cyclictest0-21swapper/023:15:240
3115072992018,1cyclictest0-21swapper/022:14:470
3115072992018,1cyclictest0-21swapper/021:13:160
3115072992018,1cyclictest0-21swapper/019:28:180
3115072992013,5cyclictest436-20systemd-journal22:09:590
3115072992013,5cyclictest436-20systemd-journal20:05:000
311507299200,19cyclictest0-21swapper/021:52:500
311507299200,19cyclictest0-21swapper/021:30:060
311507299200,19cyclictest0-21swapper/021:18:370
311507299200,19cyclictest0-21swapper/019:59:550
311507299200,19cyclictest0-21swapper/019:38:320
311507299200,18cyclictest3056-21PK-Backend22:50:090
32088902190,4sleep33056-21PK-Backend23:35:133
311508199194,3cyclictest3153178-21cstates20:55:143
3115081991917,1cyclictest3056-21PK-Backend23:26:253
3115081991917,1cyclictest3056-21PK-Backend21:25:523
3115081991917,1cyclictest0-21swapper/300:13:363
3115081991914,3cyclictest3135859-21ntp_states20:05:193
3115081991912,5cyclictest3122958-21cat19:30:133
311508199190,1cyclictest3056-21PK-Backend21:58:033
311508199190,1cyclictest3056-21PK-Backend19:56:363
311508199190,1cyclictest0-21swapper/322:34:263
311508199190,18cyclictest3056-21PK-Backend19:49:543
3115080991917,1cyclictest0-21swapper/222:19:302
3115080991917,1cyclictest0-21swapper/221:40:162
3115080991917,1cyclictest0-21swapper/220:32:232
3115080991917,1cyclictest0-21swapper/220:25:272
3115080991917,1cyclictest0-21swapper/219:13:482
3115080991914,3cyclictest3198368-21cpuspeed23:05:122
3115080991911,6cyclictest34-21ksoftirqd/219:50:122
311508099190,1cyclictest0-21swapper/222:30:592
311508099190,1cyclictest0-21swapper/222:06:532
311508099190,1cyclictest0-21swapper/221:54:082
311508099190,1cyclictest0-21swapper/200:03:062
311507699192,15cyclictest3227013-21grep00:25:221
311507699191,16cyclictest3188158-21sed22:35:181
311507299199,8cyclictest3183612-21sshd22:21:370
3115072991917,1cyclictest0-21swapper/021:35:400
3115072991917,1cyclictest0-21swapper/021:22:490
3115072991917,1cyclictest0-21swapper/020:22:260
3115072991914,3cyclictest2176-21JS10:27:480
311507299190,1cyclictest3056-21PK-Backend22:26:030
311507299190,1cyclictest3056-21PK-Backend21:04:310
3115081991816,1cyclictest0-21swapper/300:19:243
3115081991812,4cyclictest3157932-21wc21:10:013
3115080991816,1cyclictest0-21swapper/221:47:542
3115080991814,3cyclictest34-21ksoftirqd/220:50:002
3115080991811,5cyclictest3174804-21wc21:55:252
3115080991811,2cyclictest34-21ksoftirqd/223:45:192
311507299186,2cyclictest141rcu_preempt21:50:000
3115072991816,1cyclictest0-21swapper/023:28:490
3115072991816,1cyclictest0-21swapper/021:26:270
3115072991813,3cyclictest13-21ksoftirqd/023:25:010
3115072991811,5cyclictest3137073-21dpkg20:10:040
3115081991715,1cyclictest3056-21PK-Backend23:44:363
3115080991715,1cyclictest0-21swapper/220:05:402
3115080991710,6cyclictest34-21ksoftirqd/223:15:122
3115080991710,6cyclictest3162213-21memory21:20:172
3115080991710,5cyclictest3158566-21which21:10:162
311507299178,7cyclictest3219558-21cat00:05:180
311507299174,11cyclictest3217114-40logrotate00:00:010
3115081991614,1cyclictest3056-21PK-Backend20:01:573
3115081991613,2cyclictest3056-21PK-Backend21:45:463
3115081991612,3cyclictest3168313-21idleruntime-cro21:40:003
3115080991612,2cyclictest34-21ksoftirqd/222:55:032
3115080991612,2cyclictest34-21ksoftirqd/222:10:132
3115080991612,2cyclictest34-21ksoftirqd/221:00:142
3115080991612,2cyclictest34-21ksoftirqd/220:40:132
311507299160,4cyclictest3056-21PK-Backend21:05:180
139099160,4rtkit-daemon3056-21PK-Backend19:30:230
311508199157,6cyclictest3117928-21awk19:15:473
3115081991513,1cyclictest3056-21PK-Backend00:35:113
3115081991512,2cyclictest3056-21PK-Backend23:49:363
3115081991510,4cyclictest3056-21PK-Backend22:03:393
3115081991510,4cyclictest3056-21PK-Backend20:15:563
311508099157,6cyclictest3226036-21apt00:25:122
3115080991511,2cyclictest3157927-21unin-run21:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional