You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 13:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Tue Mar 31, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281073521220,4sleep10-21swapper/120:25:221
278276128140,35sleep20-21swapper/219:09:492
139099790,70rtkit-daemon0-21swapper/119:05:531
278253527244,22sleep30-21swapper/319:06:543
278243526242,14sleep00-21swapper/019:05:380
28692232430,4sleep23056-21PK-Backend23:10:232
28616102350,2sleep30-21swapper/322:50:153
28418812290,2sleep33056-21PK-Backend21:55:183
28396722260,4sleep23056-21PK-Backend21:50:122
278293499261,22cyclictest883-21dbus-daemon20:56:433
139099260,4rtkit-daemon1389-21rtkit-daemon21:30:291
28585922250,3sleep23056-21PK-Backend22:40:202
139099250,4rtkit-daemon1389-21rtkit-daemon23:40:561
139099250,4rtkit-daemon1389-21rtkit-daemon20:14:561
2782934992318,3cyclictest2845199-21df22:05:133
2782928992321,1cyclictest0-21swapper/219:54:382
28698922220,4sleep13056-21PK-Backend23:15:121
28189522220,11sleep013-21ksoftirqd/020:50:160
2782934992220,1cyclictest3056-21PK-Backend23:46:403
2782934992219,2cyclictest3056-21PK-Backend21:51:223
2782934992214,6cyclictest2812479-21python320:30:213
2782934992214,6cyclictest2791403-21python319:30:213
278293499220,8cyclictest28189512fw_packets20:50:153
2782928992220,1cyclictest0-21swapper/222:31:202
2782928992220,1cyclictest0-21swapper/222:05:592
2782928992220,1cyclictest0-21swapper/220:36:102
2782928992220,1cyclictest0-21swapper/219:38:232
2782928992220,1cyclictest0-21swapper/200:31:322
2782928992220,1cyclictest0-21swapper/200:08:142
2782927992215,5cyclictest2883509-21systemd-run23:55:001
2782927992213,7cyclictest1-21systemd00:00:001
2782920992220,1cyclictest0-21swapper/023:59:210
2782920992220,1cyclictest0-21swapper/022:34:380
2782920992220,1cyclictest0-21swapper/021:10:020
2782920992220,1cyclictest0-21swapper/020:36:500
2782920992220,1cyclictest0-21swapper/019:54:050
2782920992220,1cyclictest0-21swapper/000:22:130
278292099220,20cyclictest2896418-21apt00:30:100
139099220,4rtkit-daemon1389-21rtkit-daemon23:25:131
139099220,4rtkit-daemon1389-21rtkit-daemon23:25:121
2782934992119,1cyclictest3056-21PK-Backend20:21:103
2782934992119,1cyclictest3056-21PK-Backend19:15:343
2782934992119,1cyclictest0-21swapper/323:20:173
2782934992119,1cyclictest0-21swapper/323:18:153
2782934992119,1cyclictest0-21swapper/322:47:463
2782934992119,1cyclictest0-21swapper/321:45:263
2782934992119,1cyclictest0-21swapper/320:09:343
2782934992119,1cyclictest0-21swapper/319:40:073
2782934992113,6cyclictest2792665-21fw_packets19:35:153
278293499211,17cyclictest0-21swapper/323:30:123
278293499210,20cyclictest3056-21PK-Backend23:08:003
278293499210,20cyclictest3056-21PK-Backend21:31:473
278293499210,1cyclictest3056-21PK-Backend21:20:263
278293499210,1cyclictest3056-21PK-Backend00:23:213
278293499210,19cyclictest3056-21PK-Backend23:53:543
278293499210,19cyclictest3056-21PK-Backend22:59:483
278293499210,19cyclictest3056-21PK-Backend21:36:413
278293499210,19cyclictest3056-21PK-Backend00:37:023
278293499210,19cyclictest3056-21PK-Backend00:06:403
2782928992119,1cyclictest3056-21PK-Backend00:03:072
2782928992119,1cyclictest0-21swapper/223:00:272
2782928992119,1cyclictest0-21swapper/222:38:302
2782928992119,1cyclictest0-21swapper/221:59:512
2782928992119,1cyclictest0-21swapper/221:46:462
2782928992119,1cyclictest0-21swapper/221:30:522
2782928992119,1cyclictest0-21swapper/221:28:482
2782928992119,1cyclictest0-21swapper/221:22:052
2782928992119,1cyclictest0-21swapper/221:08:402
2782928992119,1cyclictest0-21swapper/220:59:552
2782928992119,1cyclictest0-21swapper/220:00:332
2782928992119,1cyclictest0-21swapper/219:30:052
2782928992119,1cyclictest0-21swapper/219:20:322
2782928992119,1cyclictest0-21swapper/219:15:252
2782928992119,1cyclictest0-21swapper/200:24:232
2782928992114,5cyclictest436-20systemd-journal20:29:592
278292899210,20cyclictest0-21swapper/219:55:402
278292899210,20cyclictest0-21swapper/219:41:142
278292899210,20cyclictest0-21swapper/200:10:162
278292899210,1cyclictest0-21swapper/222:01:532
2782927992118,2cyclictest3056-21PK-Backend22:33:361
2782927992117,3cyclictest1095-21runrttasks22:49:401
2782927992117,3cyclictest1095-21runrttasks19:13:351
278292799210,19cyclictest0-21swapper/121:03:451
2782920992119,1cyclictest0-21swapper/023:47:180
2782920992119,1cyclictest0-21swapper/022:03:560
2782920992119,1cyclictest0-21swapper/020:17:360
2782920992119,1cyclictest0-21swapper/019:14:230
2782920992118,2cyclictest3056-21PK-Backend22:45:400
2782920992116,3cyclictest2862687-21munin-run22:55:000
2782920992115,5cyclictest2839397-21systemd-run21:50:010
2782920992114,5cyclictest2833367-21grep21:30:170
2782920992112,7cyclictest2831282-21memory21:25:160
278292099210,20cyclictest0-21swapper/023:50:220
278292099210,20cyclictest0-21swapper/022:56:420
278292099210,20cyclictest0-21swapper/019:34:380
27913962200,4chrt3056-21PK-Backend19:30:201
278293499209,9cyclictest2850416-21entropy22:20:143
2782934992018,1cyclictest3056-21PK-Backend21:41:343
2782934992018,1cyclictest3056-21PK-Backend21:10:043
2782934992018,1cyclictest3056-21PK-Backend19:24:123
2782934992018,1cyclictest3056-21PK-Backend00:27:153
2782934992018,1cyclictest0-21swapper/322:16:163
2782934992018,1cyclictest0-21swapper/321:15:453
2782934992018,1cyclictest0-21swapper/320:35:103
2782934992013,6cyclictest2843964-21python322:00:213
2782934992012,6cyclictest436-20systemd-journal23:11:283
2782934992011,6cyclictest1-21systemd23:30:003
2782934992011,6cyclictest1-21systemd23:30:003
278293499200,1cyclictest3056-21PK-Backend23:42:353
278293499200,1cyclictest3056-21PK-Backend20:43:393
278293499200,1cyclictest3056-21PK-Backend19:46:543
278293499200,19cyclictest3056-21PK-Backend21:03:473
278293499200,19cyclictest3056-21PK-Backend00:11:353
278293499200,19cyclictest3056-21PK-Backend00:04:383
278293499200,19cyclictest0-21swapper/322:38:333
278293499200,19cyclictest0-21swapper/322:29:203
278293499200,19cyclictest0-21swapper/321:08:013
278293499200,19cyclictest0-21swapper/320:49:573
278293499200,19cyclictest0-21swapper/320:10:363
278293499200,19cyclictest0-21swapper/320:00:503
2782928992018,1cyclictest0-21swapper/222:20:052
2782928992018,1cyclictest0-21swapper/221:37:472
2782928992018,1cyclictest0-21swapper/220:51:232
2782928992018,1cyclictest0-21swapper/220:41:512
2782928992018,1cyclictest0-21swapper/220:14:122
2782928992018,1cyclictest0-21swapper/200:28:282
2782928992018,1cyclictest0-21swapper/200:15:232
2782928992017,2cyclictest0-21swapper/222:25:132
278292899200,1cyclictest3056-21PK-Backend22:54:052
278292899200,1cyclictest0-21swapper/223:53:552
278292899200,19cyclictest0-21swapper/223:41:372
278292899200,19cyclictest0-21swapper/223:29:062
278292899200,19cyclictest0-21swapper/223:29:062
278292899200,19cyclictest0-21swapper/221:03:482
278292899200,19cyclictest0-21swapper/220:47:312
278292899200,19cyclictest0-21swapper/220:19:052
278292899200,19cyclictest0-21swapper/219:14:372
278292899200,18cyclictest2848664-21df22:15:142
2782927992018,1cyclictest3056-21PK-Backend22:01:591
2782927992018,1cyclictest3056-21PK-Backend19:29:171
2782927992018,1cyclictest3056-21PK-Backend19:29:161
2782927992018,1cyclictest0-21swapper/121:35:151
2782927992018,1cyclictest0-21swapper/119:46:401
2782927992018,1cyclictest0-21swapper/100:20:291
2782927992016,2cyclictest3056-21PK-Backend20:32:091
2782927992016,2cyclictest2868151-21apt23:10:131
2782927992012,6cyclictest2863828-21awk22:55:221
278292799200,19cyclictest3056-21PK-Backend22:09:441
278292799200,19cyclictest0-21swapper/120:21:001
278292799200,19cyclictest0-21swapper/119:54:101
278292799200,19cyclictest0-21swapper/119:43:491
278292799200,19cyclictest0-21swapper/119:22:531
278292799200,19cyclictest0-21swapper/100:26:241
278292799200,18cyclictest3056-21PK-Backend22:21:321
2782920992018,1cyclictest0-21swapper/022:40:330
2782920992018,1cyclictest0-21swapper/022:22:010
2782920992018,1cyclictest0-21swapper/022:13:300
2782920992018,1cyclictest0-21swapper/021:43:210
2782920992018,1cyclictest0-21swapper/021:38:280
2782920992018,1cyclictest0-21swapper/020:41:290
2782920992018,1cyclictest0-21swapper/000:29:090
278292099200,19cyclictest0-21swapper/023:39:080
278292099200,19cyclictest0-21swapper/023:34:000
278292099200,19cyclictest0-21swapper/023:00:470
278292099200,19cyclictest0-21swapper/022:19:590
278292099200,19cyclictest0-21swapper/021:20:100
278292099200,19cyclictest0-21swapper/021:15:290
278292099200,19cyclictest0-21swapper/020:26:570
278292099200,19cyclictest0-21swapper/019:55:060
278292099200,19cyclictest0-21swapper/019:45:320
278292099200,19cyclictest0-21swapper/000:37:060
278292099200,19cyclictest0-21swapper/000:01:240
2782934991917,1cyclictest3056-21PK-Backend23:57:413
2782934991917,1cyclictest3056-21PK-Backend21:26:073
2782934991917,1cyclictest0-21swapper/320:27:403
278293499190,1cyclictest3056-21PK-Backend22:42:383
278293499190,1cyclictest0-21swapper/322:34:273
278293499190,1cyclictest0-21swapper/319:28:593
278293499190,1cyclictest0-21swapper/319:28:593
278293499190,18cyclictest3056-21PK-Backend20:15:133
278293499190,18cyclictest3056-21PK-Backend00:32:093
278293499190,18cyclictest3056-21PK-Backend00:16:273
2782928991917,1cyclictest0-21swapper/220:24:462
2782928991917,1cyclictest0-21swapper/219:45:052
2782928991916,2cyclictest3056-21PK-Backend23:06:362
2782928991912,5cyclictest2803085-21grep20:05:132
2782928991911,6cyclictest2898542-21awk00:35:162
278292899190,1cyclictest0-21swapper/223:19:542
278292899190,1cyclictest0-21swapper/222:13:562
278292899190,1cyclictest0-21swapper/220:30:282
278292899190,18cyclictest0-21swapper/223:49:482
278292899190,18cyclictest0-21swapper/221:12:322
278292899190,18cyclictest0-21swapper/219:29:032
278292899190,18cyclictest0-21swapper/219:29:032
2782927991917,1cyclictest3056-21PK-Backend22:35:391
2782927991917,1cyclictest3056-21PK-Backend20:57:161
2782927991917,1cyclictest0-21swapper/121:53:001
2782927991917,1cyclictest0-21swapper/120:17:091
2782927991917,1cyclictest0-21swapper/119:55:111
2782927991917,1cyclictest0-21swapper/100:38:141
2782927991917,1cyclictest0-21swapper/100:12:321
2782927991913,4cyclictest1095-21runrttasks22:40:141
2782927991912,5cyclictest2826211-21ntp_kernel_pll_21:10:201
2782927991911,6cyclictest2892419-21apt-get00:20:001
2782927991910,6cyclictest2839353-21cron21:50:001
278292799190,1cyclictest0-21swapper/121:40:101
278292799190,18cyclictest3056-21PK-Backend23:03:041
278292799190,18cyclictest3056-21PK-Backend19:39:091
278292799190,18cyclictest0-21swapper/121:07:371
278292799190,18cyclictest0-21swapper/100:32:201
278292799190,18cyclictest0-21swapper/100:02:311
2782920991917,1cyclictest3056-21PK-Backend20:46:090
2782920991917,1cyclictest3056-21PK-Backend20:30:340
2782920991917,1cyclictest0-21swapper/021:58:150
2782920991917,1cyclictest0-21swapper/000:10:230
2782920991913,4cyclictest2840795-21sshd21:51:190
2782920991912,5cyclictest2867956-21wc23:10:010
2782920991912,5cyclictest2809713-21wc20:25:010
278292099190,1cyclictest0-21swapper/022:29:450
278292099190,1cyclictest0-21swapper/019:21:400
278292099190,1cyclictest0-21swapper/000:19:080
278292099190,1cyclictest0-21swapper/000:07:200
278292099190,18cyclictest0-21swapper/023:42:110
278292099190,18cyclictest0-21swapper/023:17:300
278292099190,18cyclictest0-21swapper/023:12:380
278292099190,18cyclictest0-21swapper/019:44:310
2782934991816,1cyclictest3056-21PK-Backend19:52:173
2782934991816,1cyclictest3056-21PK-Backend19:12:033
2782934991811,5cyclictest2847280-21ntp_kernel_pll_22:10:203
278292899189,7cyclictest2827697-21cat21:15:172
2782928991816,1cyclictest0-21swapper/221:40:512
2782928991812,4cyclictest1095-21runrttasks22:45:202
2782928991810,6cyclictest2863684-21ntpq22:55:202
2782927991816,1cyclictest3056-21PK-Backend20:54:261
2782927991816,1cyclictest0-21swapper/123:20:131
2782927991815,2cyclictest3056-21PK-Backend21:16:091
2782927991813,3cyclictest2813555-21cstates20:35:131
2782927991811,5cyclictest2881146-21python323:45:211
2782927991810,6cyclictest2803794-21python320:05:201
278292799180,1cyclictest0-21swapper/119:15:371
278292799180,17cyclictest1389-21rtkit-daemon23:38:411
2782920991815,2cyclictest2785339-21apt19:15:110
2782920991812,4cyclictest2845535-21awk22:05:180
278292099180,1cyclictest0-21swapper/020:12:430
278292099180,1cyclictest0-21swapper/019:39:030
2782934991715,1cyclictest0-21swapper/319:58:003
2782934991714,2cyclictest3056-21PK-Backend23:35:383
2782928991710,5cyclictest2876985-21cut23:35:132
278292799172,12cyclictest2828973-21apt21:20:121
2782927991715,1cyclictest3056-21PK-Backend20:49:471
2782927991715,1cyclictest0-21swapper/123:30:131
2782927991715,1cyclictest0-21swapper/121:55:031
2782927991710,5cyclictest2888135-21cat00:05:201
278292799170,15cyclictest1389-21rtkit-daemon21:25:111
2782920991715,1cyclictest0-21swapper/021:07:110
2782920991715,1cyclictest0-21swapper/019:29:240
2782920991715,1cyclictest0-21swapper/019:29:230
2782920991712,3cyclictest2857218-21latency_hist22:40:000
278292899168,6cyclictest2875473-21awk23:30:152
278292899163,11cyclictest2871587-21munin-node23:20:092
2782928991610,4cyclictest2156-21gnome-shell23:55:092
278292799169,5cyclictest2815836-21ntpq20:40:191
278292799168,6cyclictest1-21systemd22:55:001
2782927991614,1cyclictest3056-21PK-Backend22:26:391
2782927991614,1cyclictest3056-21PK-Backend22:17:401
2782927991614,1cyclictest3056-21PK-Backend20:00:521
2782927991614,1cyclictest0-21swapper/123:07:081
278292099169,5cyclictest2874359-21grep23:25:230
278292099169,5cyclictest2874359-21grep23:25:230
278292099169,5cyclictest2823696-21cat21:05:010
2782920991614,1cyclictest0-21swapper/023:22:240
2782920991614,1cyclictest0-21swapper/020:00:470
2782920991612,3cyclictest2820843-21latency20:55:170
2782920991612,3cyclictest2803092-21cpuspeed_turbos20:05:120
2782934991510,1cyclictest3056-21PK-Backend23:00:023
278292799157,6cyclictest2846959-21awk22:10:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional