You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-08 - 05:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8s.osadl.org (updated Thu Oct 08, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168122180103,27sleep30-21swapper/319:06:103
168122180103,27sleep30-21swapper/319:06:093
169032171102,62sleep20-21swapper/219:07:212
169032171102,62sleep20-21swapper/219:07:202
167872164103,54sleep00-21swapper/019:05:510
167872164103,54sleep00-21swapper/019:05:500
167882161109,28sleep10-21swapper/119:05:521
167882161109,28sleep10-21swapper/119:05:511
1711027557,11sleep10-21swapper/119:10:001
320202620,2sleep30-21swapper/322:42:023
156612600,2sleep30-21swapper/321:41:303
247762590,2sleep30-21swapper/323:00:243
247762590,2sleep30-21swapper/323:00:243
103082560,3sleep20-21swapper/223:14:472
149552530,1sleep20-21swapper/222:30:002
53992510,2chrt0-21swapper/123:10:201
201762510,2sleep00-21swapper/023:21:120
110212510,2sleep00-21swapper/022:02:180
192622500,2sleep30-21swapper/323:45:143
192622500,2sleep30-21swapper/323:45:133
93852480,2chrt0-21swapper/323:13:523
7212470,1sleep20-21swapper/222:43:092
17381993523,6cyclictest13084-21apt-get20:15:100
1738199341,1cyclictest0-21swapper/021:07:560
248612330,2chrt0-21swapper/322:12:443
212732330,1sleep10-21swapper/121:21:371
128452330,1sleep30-21swapper/322:04:103
17381993224,6cyclictest0-21swapper/020:06:500
17381993223,1cyclictest0-21swapper/020:29:180
17381993223,1cyclictest0-21swapper/020:01:150
1738199322,27cyclictest23870-21apt-get19:25:130
1738199321,28cyclictest31221-21ntp_states19:40:190
17381993122,1cyclictest0-21swapper/020:36:430
1738199310,24cyclictest0-21swapper/020:32:590
155892310,2sleep30-21swapper/323:18:213
223562300,9sleep228-21ksoftirqd/222:35:142
1738399301,27cyclictest15224-21/usr/sbin/munin20:20:192
1738199301,27cyclictest0-21swapper/020:40:340
71982290,3sleep122-21ksoftirqd/121:11:011
1738499291,25cyclictest2089-21apt-get21:05:023
1738299291,25cyclictest9520-21ntp_states20:05:191
1738199291,26cyclictest0-21swapper/020:53:290
17384992822,4cyclictest6815-21/usr/sbin/munin20:00:153
1738399282,9cyclictest626050mandb00:00:012
1738199281,25cyclictest0-21swapper/020:46:060
1738199281,25cyclictest0-21swapper/020:46:050
17384992723,2cyclictest0-21swapper/319:45:133
17384992722,3cyclictest30868-21smartctl19:40:153
1738499271,24cyclictest0-21swapper/320:27:543
1738499271,24cyclictest0-21swapper/320:22:213
1738499271,24cyclictest0-21swapper/320:10:503
1738499271,24cyclictest0-21swapper/320:08:563
1738499271,24cyclictest0-21swapper/319:56:033
1738499271,24cyclictest0-21swapper/319:56:023
1738399270,3cyclictest4302-21apt-get19:55:102
1738399270,3cyclictest4302-21apt-get19:55:092
17382992723,2cyclictest0-21swapper/120:20:191
1738299271,2cyclictest0-21swapper/120:40:101
1738299271,25cyclictest0-21swapper/121:06:261
1738299271,25cyclictest0-21swapper/120:36:081
1738299271,24cyclictest0-21swapper/121:02:191
1738299271,24cyclictest0-21swapper/120:46:031
1738299271,24cyclictest0-21swapper/120:46:021
1738299271,24cyclictest0-21swapper/120:30:461
1738299271,24cyclictest0-21swapper/120:28:431
1738299271,24cyclictest0-21swapper/120:15:381
1738299271,24cyclictest0-21swapper/120:10:381
1738299271,24cyclictest0-21swapper/119:50:121
1738199271,24cyclictest0-21swapper/020:55:530
1738199271,24cyclictest0-21swapper/020:10:330
1738199271,24cyclictest0-21swapper/019:52:290
1738199270,24cyclictest771-21ls19:45:180
130402270,11sleep122-21ksoftirqd/100:29:071
119922270,1sleep30-21swapper/322:27:093
17384992622,2cyclictest0-21swapper/320:35:073
1738499261,24cyclictest0-21swapper/320:43:043
1738499261,23cyclictest0-21swapper/320:57:193
1738499261,23cyclictest0-21swapper/320:50:263
1738499261,23cyclictest0-21swapper/320:30:533
1738499261,23cyclictest0-21swapper/320:15:153
1738499261,23cyclictest0-21swapper/319:50:183
1738399261,23cyclictest0-21swapper/221:06:362
1738399261,23cyclictest0-21swapper/220:57:072
1738399261,23cyclictest0-21swapper/220:46:022
1738399261,23cyclictest0-21swapper/220:46:012
1738399261,23cyclictest0-21swapper/220:28:082
1738399261,23cyclictest0-21swapper/220:05:542
1738399261,23cyclictest0-21swapper/220:01:132
1738399261,23cyclictest0-21swapper/219:50:252
1738399261,23cyclictest0-21swapper/219:41:292
1738399260,24cyclictest212802chrt20:33:192
17382992624,1cyclictest0-21swapper/120:56:151
17382992613,8cyclictest0-21swapper/121:50:091
1738299261,24cyclictest0-21swapper/120:01:151
1738299261,24cyclictest0-21swapper/119:56:501
1738299261,24cyclictest0-21swapper/119:56:491
1738299261,1cyclictest0-21swapper/120:50:291
17381992624,1cyclictest0-21swapper/019:55:020
17381992624,1cyclictest0-21swapper/019:55:010
1738199261,24cyclictest0-21swapper/020:20:450
1738199261,1cyclictest0-21swapper/021:01:140
268662250,2sleep0131rcuc/021:50:140
226062250,3sleep20-21swapper/223:23:372
1738499251,23cyclictest0-21swapper/320:46:013
1738499251,23cyclictest0-21swapper/320:46:003
1738499251,1cyclictest0-21swapper/321:01:553
1738499250,23cyclictest0-21swapper/319:30:253
17383992523,1cyclictest0-21swapper/220:36:592
17383992522,2cyclictest0-21swapper/220:10:512
1738399251,23cyclictest0-21swapper/220:42:322
1738399251,1cyclictest0-21swapper/221:01:252
1738399251,1cyclictest0-21swapper/220:51:582
1738399251,17cyclictest0-21swapper/220:15:312
17382992523,1cyclictest0-21swapper/119:39:451
17382992514,7cyclictest560-21dbus-daemon19:30:011
17382992512,9cyclictest12825-21gltestperf23:40:151
1738199251,21cyclictest12468-21memory21:15:190
1738199251,10cyclictest150132grep00:05:280
113722250,2sleep30-21swapper/321:15:013
64942240,2sleep10-21swapper/121:59:301
17384992416,6cyclictest23930-21sh23:24:583
17383992422,1cyclictest0-21swapper/219:47:082
1738399241,1cyclictest0-21swapper/219:39:462
17382992422,1cyclictest0-21swapper/119:45:231
1738299240,1cyclictest0-21swapper/119:40:201
17381992410,11cyclictest29840-21cut23:05:010
17381992410,11cyclictest29840-21cut23:05:010
8232230,2sleep30-21swapper/322:19:263
43312230,9sleep012-21ksoftirqd/022:21:110
1738399233,11cyclictest318952ssh00:18:552
1738399232,8cyclictest27551-21ssh22:15:122
1738399230,8cyclictest7586-21chrt22:48:072
9012220,1sleep10-21swapper/122:43:191
275582220,1sleep20-21swapper/223:50:342
27432220,1sleep20-21swapper/223:32:322
17384992213,6cyclictest52950irq/141-enp3s0-23:30:163
17384992213,6cyclictest20612-21fw_packets00:10:193
17384992210,7cyclictest52950irq/141-enp3s0-21:45:123
17383992217,3cyclictest28-21ksoftirqd/223:00:002
1738299229,9cyclictest19005-21sensors_temp23:20:231
1738199222,14cyclictest24457-21ssh23:25:040
17381992220,1cyclictest0-21swapper/019:37:540
17381992213,7cyclictest9154-21ls00:25:230
122512220,9sleep122-21ksoftirqd/122:51:031
309132210,1sleep00-21swapper/023:53:580
244082210,2chrt0-21swapper/123:49:101
244082210,2chrt0-21swapper/123:49:091
1738499212,13cyclictest25661-21apt-get23:50:123
1738399219,10cyclictest640-21snmpd21:40:312
17383992119,1cyclictest0-21swapper/219:30:142
17383992115,4cyclictest28-21ksoftirqd/223:40:002
17383992113,6cyclictest13236-21ls23:40:182
17383992112,7cyclictest20375-21sed19:15:192
17383992110,6cyclictest4443-21diskmemload00:25:222
1738299212,13cyclictest3229-21ssh22:45:141
1738299212,13cyclictest15263-21apt-get22:30:121
17382992116,3cyclictest1392-21turbostat.cron22:19:591
17382992112,6cyclictest278-21systemd-journal19:25:001
17382992110,7cyclictest53050irq/142-enp3s0-00:35:111
17381992113,6cyclictest26006-21seq00:14:440
31422200,1sleep00-21swapper/021:32:290
31422200,1sleep00-21swapper/021:32:290
306002200,5sleep30-21swapper/300:17:383
212842200,2sleep00-21swapper/023:46:000
212842200,2sleep00-21swapper/023:45:590
181042200,4sleep1211rcuc/122:55:231
180872200,1sleep00-21swapper/021:43:560
1738499209,3cyclictest113772date22:50:233
17384992014,3cyclictest7405-21sh21:35:213
17384992013,4cyclictest15807-21threads00:30:253
17384992013,4cyclictest13903-21unixbench_multi23:40:233
17384992012,6cyclictest15684-21tune2fs22:30:143
17384992011,6cyclictest656-21find21:55:083
17384992010,6cyclictest52950irq/141-enp3s0-23:25:103
1738399209,2cyclictest141rcu_preempt21:15:212
1738399208,8cyclictest26370-21sensors_temp21:25:212
1738399207,8cyclictest28-21ksoftirqd/221:58:342
1738399206,2cyclictest141rcu_preempt22:20:202
1738399202,13cyclictest24459-21sh23:25:042
1738399201,5cyclictest0-21swapper/223:45:272
1738399201,5cyclictest0-21swapper/223:45:262
17383992015,3cyclictest26281-21dump-pmu-power00:15:002
17383992012,6cyclictest27273-21ls22:15:002
17383992012,6cyclictest26821-21diskstats21:50:122
17383992012,6cyclictest13783-21cut22:04:592
17383992012,6cyclictest12922-21cron00:05:002
17383992012,6cyclictest11360-21forks23:15:162
17383992012,5cyclictest32051-21cat21:30:132
17383992012,5cyclictest32051-21cat21:30:122
17383992010,8cyclictest4117-21ssh00:21:502
1738299203,12cyclictest19568-21apt-get19:15:121
17382992014,3cyclictest19556-21/usr/sbin/munin21:45:241
17382992013,4cyclictest636-21runrttasks00:24:261
17382992013,4cyclictest14211-21cpuspeed_turbos22:05:111
17382992012,6cyclictest16733-21seq23:19:311
1738199203,14cyclictest31728-21grep23:05:240
1738199203,13cyclictest1317-21tail21:55:130
17381992014,4cyclictest16733-21sh23:43:090
17381992013,4cyclictest12422-21sh22:27:340
17381992012,6cyclictest27918-21basename19:35:010
17381992012,6cyclictest20313-21seq21:20:400
17381992012,5cyclictest4443-21diskmemload23:58:170
17381992011,7cyclictest23629-21awk22:35:240
1738499199,7cyclictest14609-21tune2fs22:05:133
1738499191,4cyclictest13509-21sh00:29:353
17384991913,3cyclictest17238-21sed22:55:153
17384991912,5cyclictest5688-21awk23:35:153
17384991912,4cyclictest20954-21sh00:35:053
17384991912,4cyclictest12759-21sh00:04:513
17384991911,6cyclictest24973-21tr19:25:233
17384991911,6cyclictest22201-21expr22:35:143
17384991911,5cyclictest1996-21grep22:20:123
17384991911,5cyclictest12076-21cut21:15:153
17384991910,6cyclictest7751-21seq22:48:163
1738399199,8cyclictest31139-21ssh23:05:202
1738399198,2cyclictest28-21ksoftirqd/221:45:142
1738399191,5cyclictest0-21swapper/222:33:262
17383991913,4cyclictest24172-21latency23:00:182
17383991913,4cyclictest24172-21latency23:00:172
17383991911,6cyclictest24197-21df19:25:162
1738399191,16cyclictest15006-21ssh00:30:182
1738299199,7cyclictest32428-21ntpq23:30:241
1738299193,5cyclictest7417-21apt-get22:00:111
1738299193,3cyclictest141rcu_preempt23:50:131
1738299191,4cyclictest0-21swapper/100:34:461
17382991913,4cyclictest8328-21sh23:36:261
17382991912,5cyclictest26661-21/usr/sbin/munin00:15:191
17382991912,5cyclictest24687-21sensors23:00:231
17382991912,5cyclictest24687-21sensors23:00:221
17382991911,6cyclictest5149-21seq21:34:291
17382991911,6cyclictest5149-21seq21:34:281
1738299191,13cyclictest30070-21ssh23:05:121
17382991910,6cyclictest32388-21cpu23:55:101
1738199199,8cyclictest6747-21cat00:00:110
17381991913,4cyclictest20517-21open_inodes19:15:210
17381991913,4cyclictest14495-21sh22:05:140
17381991913,3cyclictest4694-21sh21:10:030
17381991912,5cyclictest30016-21seq00:17:020
17381991912,4cyclictest4443-21diskmemload22:18:520
17381991911,6cyclictest5352-21ntpq23:10:200
17381991911,6cyclictest32643-21seq22:42:390
17381991911,6cyclictest31853-21tune2fs23:30:170
17381991911,6cyclictest19403-21date19:15:010
17381991911,6cyclictest19079-21seq22:32:320
17381991911,6cyclictest15960-21seq22:54:470
17381991911,6cyclictest12415-21cron21:39:590
17381991910,7cyclictest5707-21cat23:35:150
17381991910,7cyclictest27571-21cat00:40:010
136352190,1sleep1211rcuc/121:40:151
1738499189,6cyclictest968-21ntpq23:55:193
17384991813,3cyclictest28096-21sh21:50:243
17384991813,3cyclictest22492-21sh21:22:513
17384991811,5cyclictest21587-21cat19:20:003
17384991811,4cyclictest4443-21diskmemload00:09:033
1738499181,14cyclictest24880-21ssh21:25:013
17384991810,6cyclictest2730-21awk00:20:263
1738399184,8cyclictest6076-21apt-get21:35:102
1738399182,6cyclictest19253-21ssh21:20:192
1738399182,10cyclictest9907-21ssh22:50:102
1738399181,4cyclictest0-21swapper/222:07:142
17383991813,3cyclictest22128-21fschecks_time19:20:132
17383991810,6cyclictest21425-21expr00:35:122
17382991812,4cyclictest4443-21diskmemload22:22:351
17382991812,4cyclictest15109-21sh00:05:271
17382991811,5cyclictest560-21dbus-daemon21:35:211
17382991810,6cyclictest22273-21wc22:10:231
17382991810,6cyclictest21429-21awk00:10:271
1738199182,11cyclictest3047-21apt-get22:45:030
17381991813,3cyclictest20778-21/usr/sbin/munin22:10:120
17381991812,4cyclictest21397-21sh21:45:300
17381991812,4cyclictest16820-21sh23:19:350
17381991810,6cyclictest15702-21sensors_temp00:30:250
17381991810,6cyclictest1274-21cat00:20:140
17384991712,3cyclictest28474-21sh19:35:143
17384991712,3cyclictest1139-21sh21:30:283
17384991712,3cyclictest1139-21sh21:30:273
17384991711,4cyclictest1910-21sh23:07:583
17383991711,4cyclictest5722-21sed21:10:212
1738399171,12cyclictest13392-21apt-get00:05:152
17382991711,4cyclictest53050irq/142-enp3s0-23:25:191
1738299171,11cyclictest8542-21apt-get22:25:121
1738299171,11cyclictest7143-21fschecks_time00:00:151
17382991710,5cyclictest26130-21awk21:25:201
17382991710,5cyclictest11864-21expr21:15:141
1738199178,7cyclictest17686-21ntp_states22:55:200
1738199174,3cyclictest4443-21diskmemload21:26:410
17381991713,2cyclictest21700-21/usr/sbin/munin19:20:200
1738499168,6cyclictest22189-21fw_packets19:20:133
1738499168,6cyclictest18313-21ntp_kernel_pll_19:10:193
17383991610,3cyclictest28-21ksoftirqd/219:10:242
1738299162,10cyclictest22357-21df_inode22:35:151
17382991614,1cyclictest0-21swapper/119:30:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional