You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 09:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sun Jul 19, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
42022224200,16sleep20-21swapper/219:06:222
43842174100,25sleep10-21swapper/119:08:411
1646421680,1sleep10-21swapper/123:52:111
4175216695,24sleep00-21swapper/019:05:590
2881421500,1sleep30-21swapper/300:00:593
44632139103,25sleep30-21swapper/319:09:433
79462710,2sleep30-21swapper/323:45:273
33872690,8sleep122-21ksoftirqd/100:06:031
123702630,2sleep1211rcuc/100:13:181
32792610,1sleep00-21swapper/022:54:440
242822590,1sleep30-21swapper/323:58:153
292132580,4sleep3331rcuc/300:25:243
129082560,2sleep20-21swapper/222:13:162
312472550,2sleep10-21swapper/122:02:521
63142520,1sleep10-21swapper/121:44:541
114482510,2sleep20-21swapper/200:12:232
26052480,2sleep122-21ksoftirqd/122:30:011
475999470,5cyclictest32598-21ssh22:04:142
475999451,8cyclictest14263-21chrt22:38:232
242842450,2sleep00-21swapper/023:10:130
475999441,5cyclictest32694-21ssh22:51:432
475999436,27cyclictest23817-21diskmemload00:07:022
475999421,3cyclictest21034-21ssh23:55:242
184532420,1sleep20-21swapper/220:55:192
475999410,3cyclictest11621-21ssh23:49:052
47752400,2sleep012-21ksoftirqd/022:06:520
475999401,2cyclictest0-21swapper/223:00:382
475999401,25cyclictest25701-21ssh00:23:142
475999400,3cyclictest14068-21ssh23:25:592
475999391,5cyclictest27329-21rm21:11:472
475999391,3cyclictest6023-21seq22:55:432
475999381,7cyclictest28763-21ssh22:49:302
475999381,7cyclictest19494-21sh00:18:462
475999381,7cyclictest15366-21ssh00:39:562
475999381,5cyclictest4915-21ssh22:07:002
475999381,4cyclictest3193-21ssh21:18:192
475999381,4cyclictest23817-21diskmemload00:02:342
475999380,4cyclictest13765-21ssh21:50:142
475999380,3cyclictest636-21runrttasks23:32:122
475999373,20cyclictest19517-21taskset19:41:482
475999371,7cyclictest7632-21ssh22:33:302
475999371,7cyclictest26519-21ssh22:23:242
475999371,6cyclictest27752-21sh00:25:042
475999371,6cyclictest23817-21diskmemload21:38:312
475999371,5cyclictest23817-21diskmemload21:31:282
475999371,4cyclictest213542chrt21:01:042
475999371,4cyclictest17890-21ssh22:16:292
475999371,3cyclictest8413-21ssh00:34:552
475999371,3cyclictest25878-21ssh23:10:292
475999371,3cyclictest17275-21ssh21:28:522
475999370,7cyclictest12892-21seq20:42:282
475999370,3cyclictest11100-21ssh21:24:272
475999370,3cyclictest1079-21/usr/sbin/munin23:40:252
475999361,5cyclictest15161-21chrt20:49:132
475999361,4cyclictest9119-21ssh21:45:572
475999361,3cyclictest86852sleep219:18:172
475999361,3cyclictest86852sleep219:18:162
475999361,3cyclictest258692sleep219:56:002
475999361,3cyclictest23817-21diskmemload23:21:442
475999361,3cyclictest23817-21diskmemload23:16:072
475999361,3cyclictest23817-21diskmemload23:16:072
475999361,3cyclictest237852chrt19:51:432
4759993610,15cyclictest636-21runrttasks20:26:562
475999360,4cyclictest17470-21chrt19:38:142
475999360,3cyclictest30363-21ssh23:38:552
475999360,3cyclictest22089-21ssh21:55:292
475999360,3cyclictest15643-21ssh23:51:222
475999360,3cyclictest10870-21chrt19:23:352
475999351,4cyclictest636-21runrttasks23:09:212
475999351,4cyclictest23817-21diskmemload22:25:492
475999351,2cyclictest0-21swapper/222:40:242
475999350,5cyclictest2190-21ssh21:40:442
475999342,22cyclictest0-21swapper/220:20:122
475999340,5cyclictest30245-21seq20:07:022
475999340,5cyclictest30245-21seq20:07:012
189712330,2sleep30-21swapper/321:54:063
475999322,24cyclictest30745-21apt-get20:10:132
475899320,22cyclictest20454-21sh23:55:171
228272320,2sleep00-21swapper/023:56:480
4759993110,18cyclictest2343-21cron20:20:012
475899313,5cyclictest0-21swapper/122:05:161
475899311,11cyclictest23817-21diskmemload23:27:541
476099307,10cyclictest34-21ksoftirqd/323:13:533
475999307,20cyclictest23204-21grep21:05:252
475999300,3cyclictest217052sleep219:47:252
475899304,8cyclictest20691-21taskset21:30:331
475999290,4cyclictest16937-21unixbench_multi20:50:252
475899293,10cyclictest29845-21sh00:02:031
475999287,18cyclictest14688-21sed19:30:252
475999282,21cyclictest0-21swapper/219:25:132
4759992811,15cyclictest10945-21sendmail20:40:002
475899283,9cyclictest4732-21rm22:30:361
475899281,11cyclictest16863-21ssh00:16:071
4760992715,4cyclictest34-21ksoftirqd/323:00:253
476099271,16cyclictest34-21ksoftirqd/323:43:283
475999275,19cyclictest6222-21grep19:10:242
475999272,19cyclictest0-21swapper/220:00:132
475899273,11cyclictest2703-21rm23:18:031
475899273,11cyclictest2703-21rm23:18:031
475899271,9cyclictest23817-21diskmemload00:38:031
4758992719,6cyclictest25263-21ssh21:58:411
475899270,10cyclictest23817-21diskmemload00:33:001
476099265,9cyclictest34-21ksoftirqd/322:47:013
4760992611,13cyclictest34-21ksoftirqd/321:34:413
4760992610,2cyclictest141rcu_preempt21:16:123
475999261,22cyclictest7459-21gltestperf20:30:152
475899264,9cyclictest30662-21needreboot22:50:191
475899263,15cyclictest6252-21sh22:55:581
475899262,5cyclictest15053-21apt-get22:15:101
4758992623,2cyclictest22-21ksoftirqd/119:15:021
4758992623,2cyclictest22-21ksoftirqd/119:15:011
475899261,24cyclictest0-21swapper/119:14:411
4757992624,1cyclictest0-21swapper/019:14:310
264262260,2sleep10-21swapper/123:11:011
182552260,2sleep30-21swapper/323:53:583
476099254,2cyclictest141rcu_preempt21:35:093
4760992518,5cyclictest34-21ksoftirqd/322:20:003
4760992517,6cyclictest34-21ksoftirqd/322:05:003
476099251,23cyclictest0-21swapper/319:14:543
476099251,1cyclictest0-21swapper/319:15:103
476099251,1cyclictest0-21swapper/319:15:093
475899255,5cyclictest0-21swapper/123:05:121
475899252,9cyclictest23817-21diskmemload23:38:171
475899250,15cyclictest0-21swapper/123:03:581
4757992523,1cyclictest0-21swapper/019:15:020
4757992523,1cyclictest0-21swapper/019:15:010
47702240,8sleep012-21ksoftirqd/021:43:200
476099247,5cyclictest34-21ksoftirqd/322:20:293
476099247,3cyclictest141rcu_preempt00:30:393
476099245,4cyclictest34-21ksoftirqd/300:17:263
476099244,11cyclictest34-21ksoftirqd/300:36:203
4760992418,4cyclictest34-21ksoftirqd/323:40:013
475899245,5cyclictest0-21swapper/123:30:581
475899243,6cyclictest13154-21sh21:25:211
475899242,8cyclictest23817-21diskmemload00:28:421
475899242,8cyclictest21798-21sh22:20:111
476099239,3cyclictest141rcu_preempt23:17:063
476099239,3cyclictest141rcu_preempt23:17:053
476099237,8cyclictest141rcu_preempt22:07:143
476099237,2cyclictest34-21ksoftirqd/321:21:453
476099236,3cyclictest141rcu_preempt23:07:493
476099235,5cyclictest34-21ksoftirqd/321:13:483
476099232,11cyclictest0-21swapper/322:35:453
4760992314,7cyclictest560-21dbus-daemon20:30:013
475899233,5cyclictest0-21swapper/121:10:251
475899232,6cyclictest5524-21sh23:44:431
4758992313,7cyclictest21729-21fw_packets00:20:171
4758992312,8cyclictest19044-21sensors_temp19:40:221
475899231,20cyclictest5492-21diskstats23:20:161
4757992317,4cyclictest12-21ksoftirqd/021:34:590
475799231,11cyclictest24251-21apt-get21:10:130
287052230,1sleep00-21swapper/020:05:140
287052230,1sleep00-21swapper/020:05:130
476099226,7cyclictest141rcu_preempt00:14:533
476099226,3cyclictest141rcu_preempt21:45:273
476099226,2cyclictest141rcu_preempt22:40:073
476099225,3cyclictest141rcu_preempt00:20:173
476099224,2cyclictest141rcu_preempt23:27:493
476099221,9cyclictest141rcu_preempt23:24:593
4760992213,7cyclictest9493-21perl20:35:173
4760992213,6cyclictest278-21systemd-journal22:30:003
4760992213,4cyclictest331rcuc/320:05:253
4760992213,4cyclictest331rcuc/320:05:253
475899223,6cyclictest23817-21diskmemload22:41:591
475899222,8cyclictest23817-21diskmemload22:13:031
4758992211,9cyclictest9774-21sed19:20:171
4757992216,4cyclictest29056-21awk00:25:230
4757992213,6cyclictest29426-21sed22:50:000
110572220,2sleep00-21swapper/021:47:540
476099217,3cyclictest141rcu_preempt23:33:293
476099217,2cyclictest141rcu_preempt22:59:043
4760992115,5cyclictest34-21ksoftirqd/320:35:003
4760992115,4cyclictest26188-21idleruntime-cro20:00:003
4760992115,4cyclictest23818-21idleruntime-cro21:10:013
4760992113,6cyclictest31023-21sensors_temp22:50:243
4760992112,7cyclictest3539-21ntpq20:20:213
4760992112,7cyclictest26680-21ls20:00:143
4760992112,6cyclictest22494-21df19:50:143
475899213,6cyclictest3725-21ssh21:18:501
475899212,6cyclictest22-21ksoftirqd/121:52:011
475899212,3cyclictest0-21swapper/121:20:121
4758992118,2cyclictest211rcuc/121:48:351
4758992112,7cyclictest4598-21sed20:25:011
4758992112,7cyclictest19378-21seq20:58:251
4758992111,5cyclictest21924-21apt-get21:05:121
475899210,7cyclictest23817-21diskmemload23:45:261
475799219,8cyclictest28556-21apt-get22:25:130
4757992115,5cyclictest12-21ksoftirqd/000:40:000
4757992113,6cyclictest560-21dbus-daemon00:14:590
4757992113,6cyclictest31433-21seq22:03:020
4757992113,6cyclictest20346-21smartctl21:00:160
4757992113,6cyclictest18089-21fw_packets20:55:160
4757992113,6cyclictest11548-21seq22:35:420
4757992112,7cyclictest22896-21awk00:20:260
157072210,3sleep315726-21ssh21:27:193
476099209,8cyclictest11576-21df_inode19:25:173
476099202,10cyclictest34-21ksoftirqd/321:55:513
4760992014,4cyclictest34-21ksoftirqd/321:40:003
4760992012,6cyclictest18861-21ls20:55:243
4760992012,6cyclictest1009-21find00:05:123
4760992010,8cyclictest34-21ksoftirqd/320:10:243
475899205,5cyclictest22-21ksoftirqd/120:15:351
475899205,12cyclictest23162-21cut22:45:141
475899203,6cyclictest17603-21runrttasks19:39:581
475899202,7cyclictest25394-21ssh21:35:001
475899202,4cyclictest0-21swapper/122:35:021
475899201,5cyclictest0-21swapper/119:50:211
4758992012,6cyclictest32105-21cat20:10:271
4758992012,6cyclictest27420-21pmu-power20:00:221
4758992011,7cyclictest15412-21date20:50:001
475799209,7cyclictest52850irq/140-enp3s0-23:35:150
4757992014,4cyclictest11363-21turbostat23:24:590
4757992012,6cyclictest1647-21cat20:15:230
4757992012,6cyclictest15584-21fw_packets22:15:140
4757992012,5cyclictest52850irq/140-enp3s0-23:05:120
475799201,16cyclictest28489-21latency_hist20:05:010
309612201,3sleep030980-21ssh00:03:100
476099192,3cyclictest141rcu_preempt22:32:533
476099191,9cyclictest34-21ksoftirqd/322:13:573
4760991913,4cyclictest34-21ksoftirqd/320:50:003
4760991911,6cyclictest15834-21cat20:50:153
4760991911,6cyclictest13119-21basename20:44:593
475899198,8cyclictest13142-21dump-pmu-power19:30:011
475899195,12cyclictest20466-21tune2fs19:45:171
4758991915,3cyclictest211rcuc/120:05:131
4758991915,3cyclictest211rcuc/120:05:121
4758991911,6cyclictest9393-21diskstats20:35:151
4758991911,5cyclictest14969-21seq19:31:181
4758991910,7cyclictest24509-21expr19:55:141
4758991910,7cyclictest21837-21sed21:05:011
475799192,12cyclictest9057-21apt-get20:35:140
475799192,12cyclictest10769-21ssh23:00:140
4757991914,3cyclictest23817-21diskmemload23:28:160
4757991914,3cyclictest14527-21timerwakeupswit23:50:260
4757991914,3cyclictest10274-21sh21:23:360
4757991913,4cyclictest31777-21ssh23:15:210
4757991913,4cyclictest31777-21ssh23:15:200
4757991913,4cyclictest25753-21sh22:22:370
4757991913,4cyclictest20000-21sh00:19:170
4757991913,4cyclictest1545-21sh23:40:430
4757991912,5cyclictest6164-21seq00:32:380
4757991912,5cyclictest14658-21sed19:30:250
4757991911,6cyclictest641-21seq21:15:480
4757991911,6cyclictest12366-21sensors_temp20:40:230
4757991911,5cyclictest13663-21cat20:45:130
4757991910,7cyclictest25704-21grep19:55:240
4757991910,6cyclictest20770-21cat19:45:200
475799190,5cyclictest0-21swapper/000:06:120
4760991813,3cyclictest21032-21timerandwakeup21:00:233
4760991813,3cyclictest15707-21acpi19:35:103
4760991812,4cyclictest9760-21kernelversion19:20:173
4760991811,5cyclictest18350-21tune2fs19:40:143
4760991810,6cyclictest20209-21seq19:45:153
4760991810,6cyclictest13563-21ls19:30:133
475899189,7cyclictest11281-21cat20:40:141
475899187,8cyclictest7746-21needreboot20:30:191
475899181,6cyclictest22-21ksoftirqd/120:50:261
4758991810,6cyclictest4777-21seq20:25:111
475799182,12cyclictest16276-21apt-get22:40:120
4757991813,3cyclictest5081-21ntp_states22:55:220
4757991813,3cyclictest31070-21sh20:10:150
4757991813,3cyclictest22357-21sh23:32:360
4757991813,3cyclictest15821-21df_inode20:50:140
4757991813,3cyclictest13292-21proc_pri21:25:240
4757991813,3cyclictest11405-21sh22:11:450
4757991812,4cyclictest8803-21latency_hist20:35:000
4757991811,5cyclictest5725-21pluginstate20:25:210
4757991811,5cyclictest2819-21expr20:20:130
4757991811,5cyclictest278-21systemd-journal23:50:000
4757991811,5cyclictest278-21systemd-journal22:35:000
4757991811,5cyclictest18028-21expr19:40:130
4757991810,6cyclictest13995-21forks21:50:150
259062180,2sleep00-21swapper/021:35:140
475799179,5cyclictest20404-21cpuspeed_turbos21:55:130
4757991713,2cyclictest22144-21/usr/sbin/munin19:50:180
475799171,11cyclictest9107-21apt-get19:20:110
4757991710,5cyclictest16803-21rt-features19:35:210
4757991710,5cyclictest11513-21expr19:25:160
476099169,5cyclictest541-21grep20:15:123
475799168,6cyclictest23460-21seq21:05:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional