You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 04:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Aug 11, 2026 00:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
325752175101,24sleep20-21swapper/219:08:462
325752175101,24sleep20-21swapper/219:08:452
32329217497,25sleep10-21swapper/119:05:361
32329217497,25sleep10-21swapper/119:05:351
2431321530,1sleep10-21swapper/121:12:461
307722144107,26sleep30-21swapper/319:05:093
307722144107,26sleep30-21swapper/319:05:093
324542139102,26sleep00-21swapper/019:07:110
324542139102,26sleep00-21swapper/019:07:110
2249021370,1sleep00-21swapper/000:22:430
194021030,3sleep00-21swapper/022:31:360
274792730,1sleep10-21swapper/122:26:391
80182660,10sleep122-21ksoftirqd/120:40:201
80182660,10sleep122-21ksoftirqd/120:40:201
215382620,3sleep20-21swapper/223:34:592
149042620,2sleep10-21swapper/122:17:361
149042620,2sleep10-21swapper/122:17:351
125352600,2sleep00-21swapper/000:39:580
91072540,9sleep334-21ksoftirqd/321:49:523
100122540,2sleep20-21swapper/222:14:272
106612520,1sleep00-21swapper/022:38:330
129192500,3sleep30-21swapper/323:28:043
40582490,2sleep20-21swapper/220:30:222
118372470,1sleep20-21swapper/222:39:442
483994610,11cyclictest2023-21ssh21:20:182
483994513,9cyclictest19786-21diskmemload21:18:002
483994413,9cyclictest19786-21diskmemload00:09:312
483994411,9cyclictest19786-21diskmemload21:44:372
483994314,10cyclictest417-21ssh22:06:392
483994313,11cyclictest19786-21diskmemload23:51:272
483994312,11cyclictest18926-21sh23:55:492
483994310,10cyclictest19786-21diskmemload00:32:062
156452430,2sleep0131rcuc/023:30:150
483994215,10cyclictest19786-21diskmemload23:25:572
483994213,10cyclictest19786-21diskmemload22:51:322
483994212,12cyclictest32092-21ssh23:18:212
483994112,9cyclictest19786-21diskmemload21:31:012
483994112,9cyclictest19786-21diskmemload21:31:012
483994013,9cyclictest19786-21diskmemload21:48:242
483994013,11cyclictest26397-21sh21:14:522
483994013,10cyclictest4788-21rm22:58:062
483994013,10cyclictest4788-21rm22:58:052
483994010,9cyclictest536-21ssh23:42:552
483994010,9cyclictest536-21ssh23:42:542
27122400,2sleep10-21swapper/122:56:001
27122400,2sleep10-21swapper/122:56:001
483993914,8cyclictest10364-21ssh23:01:542
483993913,11cyclictest13529-21ssh00:15:302
483993913,10cyclictest19786-21diskmemload22:01:382
483993912,9cyclictest25556-21rm00:00:402
483993912,9cyclictest1821-21taskset22:31:292
483993912,10cyclictest24638-21rm21:36:322
483993912,10cyclictest21099-21ssh21:58:232
483993912,10cyclictest19786-21diskmemload21:26:122
483993912,10cyclictest19786-21diskmemload00:13:382
483993911,10cyclictest21249-21ssh00:21:302
483993910,12cyclictest26227-21ssh23:14:152
187252390,9sleep012-21ksoftirqd/023:08:310
483993814,9cyclictest15392-21ssh22:18:052
483993814,9cyclictest15392-21ssh22:18:052
483993813,9cyclictest19786-21diskmemload23:46:492
483993813,10cyclictest19786-21diskmemload22:22:592
483993812,9cyclictest19786-21diskmemload22:25:392
483993812,9cyclictest19786-21diskmemload21:53:222
483993812,8cyclictest19786-21diskmemload22:49:162
483993713,9cyclictest24587-21rm23:36:172
483993711,11cyclictest15978-21sh22:42:092
483993710,9cyclictest16923-21ssh23:06:422
483993610,9cyclictest19786-21diskmemload23:22:442
483993511,8cyclictest10066-21ssh00:37:292
48399349,7cyclictest9145-21cat19:30:022
483993410,9cyclictest49992sleep220:35:082
483993410,8cyclictest29519-21ssh00:28:002
48399335,15cyclictest0-21swapper/219:20:132
483993210,9cyclictest15244-21expr19:40:252
481993228,3cyclictest131rcuc/023:58:350
48399319,9cyclictest1118-21fschecks_count20:25:132
483993110,8cyclictest1507-21ntpq19:10:492
48199313,7cyclictest12-21ksoftirqd/000:15:420
48199313,4cyclictest12-21ksoftirqd/021:30:010
48399309,8cyclictest0-21swapper/221:09:192
48199304,7cyclictest12-21ksoftirqd/023:54:010
147292290,3chrt0-21swapper/121:30:001
484992819,6cyclictest12389-21sh23:51:013
483992816,9cyclictest32294-21timerandwakeup20:20:252
483992813,13cyclictest27709-21grep20:10:222
48299281,25cyclictest0-21swapper/119:14:181
48199286,6cyclictest131rcuc/021:30:150
48199286,6cyclictest131rcuc/021:30:140
48199285,6cyclictest131rcuc/021:56:110
48199284,9cyclictest12-21ksoftirqd/021:54:500
481992824,3cyclictest131rcuc/000:11:130
481992811,6cyclictest12-21ksoftirqd/022:53:390
50962270,2sleep10-21swapper/121:22:131
48399276,7cyclictest0-21swapper/221:00:152
483992713,11cyclictest11285-21sed19:34:592
483992712,12cyclictest13460-21apt-config19:40:012
48199277,4cyclictest0-21swapper/000:30:450
48199274,7cyclictest12-21ksoftirqd/023:39:140
48199274,7cyclictest12-21ksoftirqd/022:01:160
48199273,8cyclictest12-21ksoftirqd/022:56:430
48199273,8cyclictest12-21ksoftirqd/022:56:420
48199273,7cyclictest12-21ksoftirqd/022:10:240
48199272,8cyclictest12-21ksoftirqd/022:40:440
481992724,2cyclictest131rcuc/022:07:400
481992724,2cyclictest131rcuc/000:06:370
48399265,8cyclictest0-21swapper/220:00:182
483992618,4cyclictest16010-21apt-get19:45:112
483992615,8cyclictest7645-21tune2fs20:40:162
483992615,8cyclictest7645-21tune2fs20:40:162
483992615,8cyclictest20683-21tune2fs19:55:152
483992615,8cyclictest19036-21awk19:50:202
483992614,9cyclictest3606-21/usr/sbin/munin19:15:232
483992614,9cyclictest29445-21/usr/sbin/munin20:15:182
482992619,5cyclictest22-21ksoftirqd/122:15:011
482992617,6cyclictest23194-21sh22:00:131
48199265,7cyclictest12-21ksoftirqd/022:48:350
48199265,6cyclictest12-21ksoftirqd/021:22:140
48199264,7cyclictest12-21ksoftirqd/022:27:540
48199264,6cyclictest12-21ksoftirqd/019:55:190
48199263,8cyclictest12-21ksoftirqd/023:18:100
48199263,7cyclictest12-21ksoftirqd/023:44:200
48199263,7cyclictest12-21ksoftirqd/023:44:190
48199263,7cyclictest12-21ksoftirqd/023:10:080
48199263,7cyclictest12-21ksoftirqd/022:17:050
48199263,7cyclictest12-21ksoftirqd/022:17:040
48199263,7cyclictest12-21ksoftirqd/000:25:200
48199263,6cyclictest12-21ksoftirqd/021:35:230
481992622,3cyclictest131rcuc/021:41:480
481992620,4cyclictest131rcuc/023:02:180
48199261,24cyclictest0-21swapper/019:13:230
211212260,2sleep10-21swapper/123:58:011
48499251,23cyclictest0-21swapper/319:13:213
483992515,7cyclictest25055-21grep20:05:162
483992514,8cyclictest13761-21expr20:55:122
483992513,9cyclictest10222-21awk20:45:222
482992517,6cyclictest19655-21sh22:20:341
48199257,7cyclictest12-21ksoftirqd/019:20:000
48199254,7cyclictest12-21ksoftirqd/023:47:580
48199254,6cyclictest12-21ksoftirqd/021:18:460
48199253,7cyclictest12-21ksoftirqd/020:30:100
48199253,6cyclictest12-21ksoftirqd/023:29:260
48199253,6cyclictest12-21ksoftirqd/000:01:260
48199252,7cyclictest12-21ksoftirqd/021:12:010
48199252,16cyclictest17799-21ssh22:20:130
48199251,6cyclictest7221-21ssh21:47:590
313322250,12sleep122-21ksoftirqd/122:05:231
48499243,16cyclictest990-21sed22:55:173
48499243,16cyclictest990-21sed22:55:163
483992415,6cyclictest11633-21/usr/sbin/munin20:50:132
48199243,7cyclictest12-21ksoftirqd/023:20:020
481992420,3cyclictest131rcuc/019:24:130
48499232,15cyclictest3147-21sh23:45:033
484992315,6cyclictest1554-21sh21:44:053
482992310,10cyclictest0-21swapper/100:10:141
48199237,5cyclictest12-21ksoftirqd/020:24:150
48199231,6cyclictest27156-21perl20:10:150
48199230,8cyclictest12-21ksoftirqd/019:25:190
48499229,11cyclictest30607-21seq22:29:463
48499221,8cyclictest13489-21idleruntime-cro23:05:013
484992214,6cyclictest29847-21ntp_states23:40:223
484992214,6cyclictest29847-21ntp_states23:40:213
484992213,7cyclictest560-21dbus-daemon22:04:593
484992212,6cyclictest6422-21sort22:35:153
484992211,9cyclictest28341-21cat23:39:593
48299221,6cyclictest0-21swapper/123:22:121
482992214,6cyclictest19964-21seq23:33:231
482992212,8cyclictest560-21dbus-daemon23:50:011
48299220,6cyclictest7045-21ssh22:35:211
48199223,6cyclictest12-21ksoftirqd/020:41:390
48199223,6cyclictest12-21ksoftirqd/020:41:380
48199223,12cyclictest0-21swapper/020:01:320
48199222,6cyclictest12-21ksoftirqd/019:35:140
481992212,7cyclictest25867-21cat20:05:240
195132220,2sleep30-21swapper/323:09:183
74622210,1sleep30-21swapper/320:40:143
74622210,1sleep30-21swapper/320:40:143
48499213,11cyclictest27245-21ssh21:15:163
48499211,9cyclictest30474-21grep22:05:173
484992115,2cyclictest0-21swapper/300:35:143
484992113,6cyclictest22827-21ntpq21:35:203
484992113,6cyclictest14513-21ntp_states20:55:193
484992112,6cyclictest278-21systemd-journal19:30:053
484992112,6cyclictest18950irq/143-ahci[0000:14:553
484992111,6cyclictest52950irq/141-enp3s0-22:45:153
484992110,6cyclictest52950irq/141-enp3s0-22:10:113
484992110,3cyclictest141rcu_preempt00:10:013
48299217,5cyclictest8788-21expr23:25:141
482992113,6cyclictest7534-21diskstats23:00:141
482992113,6cyclictest17761-21seq19:48:271
482992113,6cyclictest10685-21smartctl23:50:161
482992112,7cyclictest18599-21sh21:32:161
482992112,7cyclictest18599-21sh21:32:151
48199213,3cyclictest12-21ksoftirqd/020:15:170
48199213,10cyclictest12-21ksoftirqd/019:30:190
481992113,6cyclictest15198-21seq20:56:280
244932210,1sleep10-21swapper/122:48:551
48499206,10cyclictest29942-21apt00:05:003
48499203,10cyclictest1595-21ssh21:20:143
48499202,12cyclictest16010-21apt-get19:45:123
484992015,3cyclictest17335-21run-parts22:20:003
484992015,3cyclictest17335-21run-parts22:20:003
484992014,4cyclictest31333-21sh22:53:583
484992014,4cyclictest22179-21sendmail20:00:003
484992013,4cyclictest636-21runrttasks23:22:563
484992012,6cyclictest5740-21sed20:35:183
484992012,6cyclictest16001-21cat21:00:123
484992011,7cyclictest18751-21cat19:50:183
484992010,7cyclictest22527-21sort20:00:133
48299206,4cyclictest66050lldpd19:22:051
48299202,12cyclictest75-21kswapd000:05:161
482992012,6cyclictest10702-21ls20:45:281
482992012,6cyclictest10304-21open_inodes19:30:221
48299201,14cyclictest27017-21sh21:15:031
482992010,6cyclictest52850irq/140-enp3s0-21:55:101
48199204,6cyclictest12-21ksoftirqd/020:26:030
48199204,6cyclictest12-21ksoftirqd/019:53:420
481992016,3cyclictest131rcuc/020:49:370
481992012,6cyclictest6291-21sed20:35:250
48199200,7cyclictest13295-21taskset20:54:310
48199200,7cyclictest12-21ksoftirqd/021:00:220
48199200,7cyclictest12-21ksoftirqd/019:45:460
136632200,1sleep30-21swapper/321:52:413
48499193,9cyclictest28859-21apt-get20:15:153
48499191,8cyclictest3476-21runrttasks22:33:073
484991912,5cyclictest16321-21ntpq23:30:213
484991912,5cyclictest13445-21sed20:55:003
484991910,6cyclictest12920-21grep19:35:233
48299196,6cyclictest27844-21cat23:15:181
48299191,4cyclictest0-21swapper/123:41:241
48299191,4cyclictest0-21swapper/123:41:241
482991914,3cyclictest28095-21sh22:50:451
482991914,3cyclictest23835-21sh00:24:041
482991913,4cyclictest29918-21swap21:40:231
482991912,5cyclictest19916-21seq19:53:411
482991911,6cyclictest32637-21processes22:30:211
482991911,6cyclictest30166-21ls20:15:261
482991911,6cyclictest2567-21seq20:29:201
482991911,6cyclictest22293-21unixbench_singl23:10:251
482991911,6cyclictest20258-21seq19:55:101
482991911,6cyclictest16870-21seq21:00:211
482991911,6cyclictest14365-21seq00:16:221
482991911,5cyclictest14208-21diskstats19:40:151
482991910,7cyclictest11736-21seq00:39:081
48199191,5cyclictest12-21ksoftirqd/021:10:010
481991910,7cyclictest14269-21tune2fs19:40:160
48499189,6cyclictest14340-21grep19:40:173
48499188,6cyclictest9667-21switchtime21:25:233
48499187,7cyclictest4878-21apt-get19:20:143
48499183,9cyclictest18536-21ssh00:20:163
48499182,12cyclictest28517-21ntp_states23:15:233
48499182,10cyclictest15110-21ssh21:30:143
48499182,10cyclictest15110-21ssh21:30:143
484991813,3cyclictest17985-21cstates22:20:143
484991812,4cyclictest445-21idleruntime-cro20:24:593
484991812,4cyclictest20422-21sh21:57:433
484991812,4cyclictest19786-21diskmemload21:11:133
484991811,4cyclictest19786-21diskmemload23:10:203
484991811,4cyclictest19786-21diskmemload22:40:243
484991810,6cyclictest3090-21cat20:30:123
484991810,6cyclictest25521-21awk20:05:213
484991810,6cyclictest1495-21ls20:25:173
48499180,5cyclictest0-21swapper/320:13:253
48299182,11cyclictest17901-21apt-get21:05:141
482991813,3cyclictest7417-21sort19:25:141
482991813,3cyclictest27808-21sh00:26:171
482991812,4cyclictest13207-21sh22:40:171
482991811,5cyclictest5446-21cut20:35:151
482991811,5cyclictest52850irq/140-enp3s0-00:30:011
482991811,5cyclictest3635-21cat21:45:191
482991811,5cyclictest3236-21forks20:30:141
482991811,5cyclictest26135-21seq20:05:531
482991811,5cyclictest22880-21awk21:35:231
482991811,5cyclictest22246-21expr23:35:131
482991811,5cyclictest15319-21seq20:58:021
48299181,14cyclictest52850irq/140-enp3s0-23:05:131
482991811,4cyclictest19786-21diskmemload00:02:541
482991811,4cyclictest10318-21idleruntime21:50:171
482991810,6cyclictest28740-21cut20:15:011
180672180,2sleep30-21swapper/321:05:143
48499176,7cyclictest953-21cat00:30:233
48499173,12cyclictest17326-21tail23:55:143
484991710,5cyclictest7743-21sed19:25:173
484991710,5cyclictest10629-21sed20:45:263
484991710,4cyclictest19786-21diskmemload00:17:003
48299172,11cyclictest11462-21apt-get20:50:131
482991712,3cyclictest3400-21latency19:15:201
482991710,5cyclictest24264-21seq20:04:201
482991710,5cyclictest12423-21uname19:35:191
48499168,6cyclictest3068-21awk19:15:173
48499161,11cyclictest25581-21ssh00:25:143
482991511,2cyclictest31627-21if_enp3s020:20:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional