You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 03:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu May 28, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
89302174102,27sleep30-21swapper/319:07:293
809521650,3sleep20-21swapper/219:05:192
89052164100,57sleep00-21swapper/019:07:100
88042160108,28sleep10-21swapper/119:05:511
596421250,1sleep10-21swapper/121:16:031
312082760,1sleep20-21swapper/200:24:062
20122670,2sleep228-21ksoftirqd/222:50:152
211232650,3chrt22-21ksoftirqd/123:52:191
226482580,1sleep00-21swapper/022:40:500
40112490,1sleep30-21swapper/322:51:123
88722480,2sleep30-21swapper/300:30:443
183592470,2sleep3331rcuc/323:25:513
83192460,2sleep20-21swapper/220:20:182
226462450,1sleep10-21swapper/123:05:191
211612450,1sleep30-21swapper/323:52:203
173862410,1sleep20-21swapper/220:40:252
207342400,1sleep20-21swapper/223:04:272
35102390,1sleep20-21swapper/200:03:262
81822330,2chrt0-21swapper/221:18:172
276632320,2chrt0-21swapper/123:33:281
939899311,27cyclictest18200-21ssh00:14:323
9397993117,8cyclictest29691-21sh23:35:042
144782310,11chrt12-21ksoftirqd/022:35:160
190282300,3chrt0-21swapper/100:39:131
9398992922,4cyclictest3408-21appstreamcli21:15:013
939599291,26cyclictest0-21swapper/020:05:190
939799281,3cyclictest3007-21apt-config21:15:002
9396992824,2cyclictest10974-21hddtemp_smartct21:20:151
93562280,2chrt0-21swapper/000:31:130
325162280,1sleep00-21swapper/021:12:090
9397992722,3cyclictest549-21apt-get20:05:022
9397992722,3cyclictest28-21ksoftirqd/219:40:002
939799271,24cyclictest0-21swapper/220:02:412
9396992723,2cyclictest28652-21diskmemload00:15:151
939699271,24cyclictest0-21swapper/120:05:031
9395992711,9cyclictest32549-21runrttasks00:25:110
48292270,2chrt0-21swapper/022:52:010
939899261,23cyclictest0-21swapper/321:20:163
939799262,17cyclictest8242-21ssh22:55:122
939699261,24cyclictest0-21swapper/119:54:261
939699260,24cyclictest0-21swapper/120:00:181
939599261,24cyclictest0-21swapper/021:22:240
939599261,24cyclictest0-21swapper/021:17:100
939599261,1cyclictest0-21swapper/020:00:140
211532260,2chrt0-21swapper/221:51:242
939899251,1cyclictest0-21swapper/320:00:033
9398992510,12cyclictest5381-21ssh23:16:213
939899250,2cyclictest1859-21ls20:05:183
939799252,8cyclictest9444-21ssh23:20:102
9397992512,6cyclictest28652-21diskmemload00:10:112
165172250,2sleep00-21swapper/023:00:240
9398992422,1cyclictest0-21swapper/300:15:183
9398992415,4cyclictest0-21swapper/322:20:103
9398992414,7cyclictest27576-21df22:45:133
9396992414,8cyclictest18158-21seq20:45:041
9396992414,7cyclictest14414-21fw_packets19:20:151
9395992422,1cyclictest0-21swapper/000:15:200
9395992416,6cyclictest22340-21hddtemp_smartct23:05:150
325372240,2sleep10-21swapper/121:35:451
253752240,3sleep20-21swapper/221:55:132
135562240,2sleep20-21swapper/200:35:132
9398992318,3cyclictest15151-21/usr/sbin/munin19:20:233
939899231,19cyclictest5630-21df20:15:133
9397992316,4cyclictest20068-21cron20:50:002
9396992314,6cyclictest15048-21sensors_temp20:35:221
9395992315,6cyclictest14568-21munin-run23:00:000
9395992310,11cyclictest27292-21ssh21:55:510
195312230,3sleep20-21swapper/222:39:272
118382230,1sleep20-21swapper/219:15:142
9398992213,7cyclictest8424-21sensors_temp22:30:223
9398992213,7cyclictest25130-21fw_packets19:45:153
9398992213,6cyclictest13809-21apt22:35:103
9398992212,7cyclictest14589-21latency20:35:173
9397992213,6cyclictest560-21dbus-daemon19:30:002
939799221,18cyclictest6281-21sed00:05:162
9396992216,4cyclictest22-21ksoftirqd/100:15:001
9396992215,4cyclictest9582-21sh23:44:091
9396992212,7cyclictest7407-21cat22:30:121
9396992212,7cyclictest560-21dbus-daemon00:00:001
939699221,18cyclictest25811-21grep22:20:011
939599222,13cyclictest31832-21ssh22:00:110
73022220,1sleep20-21swapper/221:41:122
204872221,1sleep10-21swapper/121:27:081
939899219,5cyclictest32432-21ssh00:25:073
939899211,9cyclictest0-21swapper/322:42:553
9398992113,6cyclictest23418-21pluginstate20:55:203
9398992113,6cyclictest19566-21ls20:45:233
9398992112,7cyclictest5673-21cat00:05:113
9398992112,6cyclictest31581-21cat22:00:013
9398992112,6cyclictest22407-21cut20:55:013
9398992111,7cyclictest278-21systemd-journal20:15:003
9398992111,7cyclictest18239-21grep19:30:133
9397992119,1cyclictest0-21swapper/221:21:082
9397992113,6cyclictest23127-21ls23:05:232
9397992112,6cyclictest27193-21ntpq22:20:202
9397992111,7cyclictest26085-21awk19:45:242
939799210,5cyclictest0-21swapper/222:29:092
9396992115,4cyclictest10592-21sh22:08:121
9396992114,4cyclictest31282-21sh21:10:541
9396992113,6cyclictest27845-21seq00:20:421
9396992113,6cyclictest2460-21seq22:01:451
9396992112,6cyclictest31444-21seq22:00:001
9396992111,7cyclictest20000-21timerwakeupswit21:50:231
939599212,13cyclictest5706-21ssh00:05:120
9395992114,6cyclictest12-21ksoftirqd/023:50:010
9395992112,6cyclictest21490-21python320:50:230
939599211,17cyclictest28652-21diskmemload23:13:270
318252210,2sleep10-21swapper/123:35:401
179282211,17sleep018068-21ntpq21:25:180
939899204,12cyclictest11581-21ssh21:45:033
9398992013,5cyclictest53050irq/142-enp3s0-23:05:003
9398992013,4cyclictest28730-21sh23:57:583
9398992012,6cyclictest23230-21seq21:53:313
9398992012,6cyclictest17294-21threads20:40:243
9398992012,5cyclictest31422-21expr00:00:143
9398992012,5cyclictest2979-21apt-get21:15:003
9398992011,6cyclictest26114-21expr00:20:133
939799202,12cyclictest28652-21diskmemload23:25:102
9397992014,3cyclictest25575-21ls23:55:232
9397992013,4cyclictest8280-21processes22:30:212
9397992012,6cyclictest7381-21date20:20:012
9397992012,6cyclictest7014-21sed22:05:192
9397992012,6cyclictest23965-21awk19:40:242
9397992011,6cyclictest5173-21date20:15:002
9397992011,6cyclictest18059-21basename21:50:002
939799201,13cyclictest20541-21ssh22:40:122
939699204,13cyclictest28187-21needreboot22:45:191
939699202,12cyclictest16088-21apt-get19:25:101
939699201,6cyclictest32532-21apt-get00:25:111
9396992014,3cyclictest10402-21sh00:32:161
9396992013,5cyclictest1934-21expr22:50:141
9396992013,4cyclictest6000-21sh23:16:591
9396992012,6cyclictest8761-21seq00:06:451
9396992012,6cyclictest4804-21seq20:11:541
9396992012,6cyclictest23424-21ntp_kernel_pll_19:40:191
9396992012,6cyclictest21555-21ntpq22:40:201
9396992012,5cyclictest17213-21ntp_offset23:25:191
939599203,14cyclictest14710-21wc00:35:230
939599202,6cyclictest25824-21apt-get00:20:110
939599202,3cyclictest141rcu_preempt23:40:120
9395992015,3cyclictest18416-21missed_timers23:50:180
9395992014,4cyclictest12-21ksoftirqd/022:20:000
9395992013,5cyclictest9498-21ssh22:31:040
9395992013,5cyclictest19553-21cut21:50:200
9395992013,4cyclictest28652-21diskmemload23:34:400
9395992012,6cyclictest24430-21sed21:30:160
9395992011,7cyclictest10003-21cut20:25:130
9395992011,6cyclictest278-21systemd-journal23:20:000
73122201,2sleep37331-21ssh23:41:533
9398991914,3cyclictest21700-21sh23:05:103
9398991914,3cyclictest12761-21sh23:21:593
9398991913,4cyclictest9231-21sh22:06:503
9398991913,4cyclictest7445-21/usr/sbin/munin20:20:243
9398991913,4cyclictest3550-21sh21:39:113
9398991912,5cyclictest29255-21cat19:55:123
9398991912,5cyclictest26686-21sed21:05:013
9398991912,4cyclictest15148-21sh23:48:013
9398991912,4cyclictest15023-21sh22:10:553
939899191,16cyclictest5695-21cat21:40:183
9398991911,5cyclictest23532-21sh23:30:153
9398991911,5cyclictest17180-21which19:25:223
9398991910,7cyclictest29900-21date23:10:243
9398991910,7cyclictest20137-21seq21:26:453
9398991910,6cyclictest10434-21date20:25:173
939799199,7cyclictest27724-21ntp_states19:50:202
939799191,7cyclictest24916-21tr21:30:212
9397991911,6cyclictest593-21tune2fs00:25:152
9397991911,6cyclictest278-21systemd-journal20:35:002
9397991911,6cyclictest21866-21seq21:28:302
9397991911,6cyclictest19764-21expr22:15:132
9397991911,6cyclictest15899-21cut20:40:002
9397991911,5cyclictest5691-21cron22:05:002
9397991911,5cyclictest28652-21diskmemload00:17:452
9397991911,5cyclictest27249-21grep21:05:152
9397991911,5cyclictest23850-21grep20:55:242
9397991911,5cyclictest11685-21grep20:30:012
939799191,13cyclictest22914-21chrt23:30:122
939699197,8cyclictest22-21ksoftirqd/120:20:101
939699196,3cyclictest141rcu_preempt20:20:001
9396991914,3cyclictest9694-21sh21:43:361
9396991914,3cyclictest29431-21sh22:21:591
9396991914,3cyclictest29309-21missed_timers23:10:181
9396991914,3cyclictest2804-21sh22:26:041
9396991913,4cyclictest13728-21sh23:46:341
9396991913,3cyclictest14123-21sh21:46:051
9396991912,5cyclictest30643-21seq19:57:451
9396991912,5cyclictest27066-21seq21:31:591
9396991912,5cyclictest24964-21diskstats21:00:151
9396991911,6cyclictest26506-21seq19:49:351
9396991911,6cyclictest21456-21which20:50:221
9396991911,6cyclictest15569-21seq22:11:281
9396991911,6cyclictest12683-21seq23:21:551
9396991911,6cyclictest11483-21seq20:29:551
9396991911,5cyclictest15862-21seq22:35:441
9396991910,7cyclictest26882-21cut21:05:121
939599192,12cyclictest24666-21apt-get19:45:120
9395991913,4cyclictest2419-21sh22:25:410
9395991912,4cyclictest26718-21cut22:20:150
939599191,16cyclictest20067-21cron20:50:000
9395991911,6cyclictest13965-21cat20:35:130
9395991911,6cyclictest12066-21expr20:30:140
9395991911,5cyclictest20972-21cat19:35:170
9395991911,5cyclictest11314-21cron19:15:000
9395991911,4cyclictest12433-21cpu00:10:120
939599191,13cyclictest29223-21apt-get23:35:040
9395991910,5cyclictest28652-21diskmemload22:10:220
939899188,6cyclictest53050irq/142-enp3s0-22:15:123
9398991813,3cyclictest3295-21sh22:02:363
9398991812,4cyclictest31798-21sh23:35:403
9398991811,5cyclictest23841-21ls21:30:123
9398991811,5cyclictest1234-21sed22:25:173
9398991811,4cyclictest28652-21diskmemload22:56:333
9398991811,4cyclictest13361-21sh00:35:023
9398991810,6cyclictest28800-21wc21:10:013
9398991810,6cyclictest13768-21cat20:35:013
9398991810,5cyclictest20836-21grep19:35:163
939799183,9cyclictest6845-21ssh00:30:122
939799181,8cyclictest28765-21df_inode23:10:132
9397991812,4cyclictest28652-21diskmemload23:42:422
9397991812,4cyclictest17435-21sh22:13:222
9397991812,3cyclictest11890-21ntp_kernel_pll_23:45:202
9397991811,5cyclictest9617-21expr19:10:132
939799181,14cyclictest17911-21diskstats23:50:142
9396991812,4cyclictest31182-21/usr/sbin/munin00:00:231
9396991811,5cyclictest22998-21expr20:55:151
9396991811,5cyclictest11055-21seq22:56:311
9396991810,6cyclictest21370-21which19:35:221
9396991810,6cyclictest16288-21run-parts23:00:221
939599182,9cyclictest30498-21ssh21:35:130
939599182,3cyclictest141rcu_preempt20:10:110
939599181,6cyclictest73042ssh21:41:120
939599181,6cyclictest28655-21dump-pmu-power21:10:000
9395991813,3cyclictest31842-21sh00:00:160
9395991812,4cyclictest24435-21/usr/sbin/munin23:55:250
9395991811,5cyclictest8914-21awk20:20:240
9395991811,5cyclictest27505-21sed19:50:160
9395991811,5cyclictest15310-21sed19:20:240
9395991811,4cyclictest28652-21diskmemload21:45:140
939599181,14cyclictest17363-21wc23:25:210
9395991810,6cyclictest28805-21ls22:45:240
9395991810,6cyclictest19323-21which19:30:230
9395991810,6cyclictest17352-21sed20:40:240
9395991810,6cyclictest10263-21cat23:20:160
939899179,6cyclictest11334-21ls19:15:003
939899172,11cyclictest22527-21apt-get19:40:123
9398991711,4cyclictest12435-21netstat19:15:203
939899171,11cyclictest26802-21apt-get19:50:113
939799176,9cyclictest0-21swapper/223:15:172
9397991712,3cyclictest28917-21/usr/sbin/munin19:55:182
9397991712,3cyclictest24603-21/usr/sbin/munin21:00:142
9397991711,4cyclictest32214-21sh22:48:442
9397991710,5cyclictest31579-21ntpq21:35:222
9397991710,5cyclictest18451-21expr19:30:142
939699179,6cyclictest18138-21cut20:45:001
939699178,7cyclictest13469-21seq20:32:561
9396991711,3cyclictest18303-21cpuspeed_turbos19:30:121
939699171,11cyclictest9437-21apt-get19:10:121
939599178,6cyclictest29342-21cut19:55:140
939599172,11cyclictest11580-21apt-get19:15:130
9395991712,3cyclictest22934-21grep20:55:150
9395991710,4cyclictest28652-21diskmemload22:06:310
939799168,6cyclictest15977-21wc19:25:012
9396991611,3cyclictest11555-21/usr/sbin/munin19:15:191
939599169,5cyclictest22796-21cat19:40:130
939599168,6cyclictest25900-21sed21:00:250
9395991611,3cyclictest6263-21pmu-power20:15:210
9395991611,3cyclictest16495-21fschecks_time19:25:130
9397991510,3cyclictest21251-21ntp_states20:50:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional