You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 12:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Mon Dec 22, 2025 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3508812890,6chrt0-21swapper/321:10:003
3088832660,3sleep20-21swapper/220:29:022
230042995437,13cyclictest292002-21cat20:10:001
230040995242,7cyclictest278-21systemd-journal00:00:000
230040995042,6cyclictest237167-21kworker/u8:3+flush-8:021:49:060
230040994941,6cyclictest390812-21kworker/u8:0+flush-8:022:19:150
230040994844,3cyclictest390812-21kworker/u8:0+events_unbound22:00:010
230040994836,10cyclictest360755-21apt-get21:20:000
230040994836,10cyclictest360755-21apt-get21:20:000
230040994740,5cyclictest390812-21kworker/u8:0+events_unbound22:15:000
230042994527,15cyclictest480469-21/usr/sbin/munin23:24:041
230040994540,3cyclictest450195-21kworker/u8:0+events_unbound23:04:120
230040994539,4cyclictest425332-21kworker/u8:1+events_unbound23:09:110
230040994438,4cyclictest450195-21kworker/u8:0+flush-8:022:58:590
230040994336,5cyclictest547595-21kworker/u8:2+flush-8:000:29:110
230040994136,4cyclictest215362-21kworker/u8:1+events_unbound20:24:120
230045994036,3cyclictest499904-21kworker/u8:3+events_unbound23:45:002
230042994034,4cyclictest237167-21kworker/u8:3+flush-8:019:43:571
230042994032,5cyclictest281771-21latency_hist20:00:001
23004299401,27cyclictest311020-21/usr/sbin/munin20:29:111
230040994034,4cyclictest400561-21kworker/u8:2+events_unbound22:33:560
230040994034,4cyclictest400561-21kworker/u8:2+events_unbound22:33:560
230040994033,5cyclictest198183-21kworker/u8:0+events_unbound19:15:030
230042993916,10cyclictest909-21mta-sts-daemon20:48:391
230040993934,3cyclictest215362-21kworker/u8:1+events_unbound22:05:010
230040993931,6cyclictest499904-21kworker/u8:3+flush-8:023:44:090
23004099390,37cyclictest246872-21munin-run19:25:000
230042993832,4cyclictest495284-21apt-get23:38:561
230042993826,9cyclictest545549-21irqrtprio00:29:031
230042993822,14cyclictest237733-21diskstats19:18:581
230040993834,3cyclictest425332-21kworker/u8:1+events_unbound22:29:110
230040993829,7cyclictest479052-21ntpq23:19:060
23004099380,37cyclictest365554-21sshd21:24:100
230045993733,3cyclictest215362-21kworker/u8:1+events_unbound19:39:142
230042993732,3cyclictest333905-21kthreadcore20:54:041
230042993729,6cyclictest365283-21ntpq21:24:041
230042993726,8cyclictest513789-21ntpq23:54:071
23004299370,23cyclictest576-21runrttasks23:58:561
230040993731,4cyclictest514449-21latency_hist23:55:000
230040993731,4cyclictest237167-21kworker/u8:3+events_unbound21:39:150
23004099371,33cyclictest291664-21munin-run20:10:000
230042993629,4cyclictest261590-21latency_hist19:40:011
230042993624,9cyclictest454878-21apt-get22:55:011
23004299361,19cyclictest316375-21runrttasks20:34:291
23004299360,22cyclictest440415-21runrttasks22:41:161
23004299360,22cyclictest292371-21/usr/sbin/munin20:14:041
23004299360,21cyclictest554455-21runrttasks00:36:241
23004299360,21cyclictest366910-21if_enp3s021:29:011
23004299360,21cyclictest247843-21if_enp4s0.219:29:011
230040993631,4cyclictest237167-21kworker/u8:3+flush-8:020:49:120
230042993523,10cyclictest459350-21needreboot22:59:061
23004299351,20cyclictest342041-21if_enp4s0.221:04:001
23004299350,20cyclictest499949-21runrttasks23:41:061
230040993531,3cyclictest237167-21kworker/u8:3+flush-8:020:44:120
230040993531,3cyclictest215362-21kworker/u8:1+events_unbound21:34:120
230040993529,4cyclictest539361-21kworker/u8:1+flush-8:000:34:030
230040993529,4cyclictest215362-21kworker/u8:1+flush-8:021:04:140
230040993524,9cyclictest215362-21kworker/u8:1+events_unbound21:10:000
230042993417,14cyclictest301678-21latency_hist20:20:011
23004299340,21cyclictest356107-21taskset21:15:331
23004299340,21cyclictest356107-21taskset21:15:331
23004299340,20cyclictest529798-21runrttasks00:11:041
23004299340,20cyclictest436144-21if_enp4s022:39:011
23004299340,19cyclictest425724-21runrttasks22:27:241
230040993429,4cyclictest215362-21kworker/u8:1+events_unbound21:25:070
230040993427,5cyclictest450195-21kworker/u8:0+flush-8:022:54:000
230040993427,5cyclictest450195-21kworker/u8:0+flush-8:000:24:100
230040993427,5cyclictest237167-21kworker/u8:3+flush-8:019:44:060
230040993426,6cyclictest499904-21kworker/u8:3+events_unbound00:14:080
23004299335,16cyclictest0-21swapper/121:54:171
230042993322,9cyclictest0-21swapper/123:18:571
23004299330,19cyclictest386206-21runrttasks21:46:191
23004299330,19cyclictest381305-21runrttasks21:41:251
230040993328,3cyclictest474805-21cron23:15:000
230040993328,3cyclictest215362-21kworker/u8:1+events_unbound21:44:140
230042993224,6cyclictest489215-21ntpq23:29:081
23004299322,20cyclictest0-21swapper/121:10:121
23004299322,20cyclictest0-21swapper/121:10:121
230042993221,9cyclictest317571-21grep20:39:041
230042993214,15cyclictest553588-21ntpq00:34:111
23004299320,19cyclictest415877-21runrttasks22:17:411
230040993228,3cyclictest450195-21kworker/u8:0+flush-8:023:33:580
230040993226,4cyclictest237167-21kworker/u8:3+events_unbound20:59:090
230040993224,6cyclictest520173-21/usr/sbin/munin00:04:070
230045993126,4cyclictest215362-21kworker/u8:1+flush-8:021:09:102
23004299314,16cyclictest232-21jbd2/sda2-819:21:141
230042993127,2cyclictest447727-21kthreadcore22:49:021
230042993124,5cyclictest346491-21apt-get21:08:551
230042993120,9cyclictest400458-21unixbench_multi21:59:111
230042993118,11cyclictest525134-21apt-get00:08:541
23004299310,18cyclictest494714-21runrttasks23:34:571
230045993026,3cyclictest215362-21kworker/u8:1+events_unbound22:04:042
230045993025,3cyclictest215362-21kworker/u8:1+events_unbound21:55:072
23004299302,15cyclictest0-21swapper/120:00:101
23004299301,24cyclictest505153-21apt-get23:48:571
23004299301,24cyclictest505153-21apt-get23:48:561
23004299301,24cyclictest272444-21apt-get19:53:531
23004299301,23cyclictest465528-21apt-get23:08:531
230040993026,3cyclictest490112-21kworker/u8:2+events_unbound00:05:380
230040993026,3cyclictest400561-21kworker/u8:2+events_unbound22:24:140
230040993026,3cyclictest215362-21kworker/u8:1+events_unbound22:05:070
230040993025,3cyclictest215362-21kworker/u8:1+flush-8:020:54:140
230040993024,4cyclictest444948-21kworker/u8:3+flush-8:023:20:050
23004099300,28cyclictest235037-21munin-run19:15:010
230045992925,3cyclictest400561-21kworker/u8:2+events_unbound22:44:122
230045992925,2cyclictest237167-21kworker/u8:3+events_unbound20:10:022
230045992921,6cyclictest278-21systemd-journal21:18:102
230045992921,6cyclictest278-21systemd-journal21:18:102
230042992922,5cyclictest237167-21kworker/u8:3+flush-8:020:44:001
23004299292,25cyclictest0-21swapper/120:55:201
23004299292,23cyclictest411-21dbus-daemon19:32:371
23004299292,22cyclictest416-21mta-sts-daemon22:14:091
23004299292,21cyclictest406123-21apt-get22:08:541
23004299292,20cyclictest421rcuog/200:22:231
230042992918,9cyclictest0-21swapper/122:24:131
230042992917,10cyclictest0-21swapper/119:48:551
23004299290,17cyclictest376593-21runrttasks21:38:571
230040992925,3cyclictest272125-21kworker/u8:0+flush-8:020:10:050
230040992924,4cyclictest400561-21kworker/u8:2+flush-8:022:44:160
2296992293,8sleep20-21swapper/219:09:502
230045992824,2cyclictest312354-21fschecks_time20:33:562
230042992820,6cyclictest278-21systemd-journal21:32:101
230042992819,6cyclictest278-21systemd-journal22:02:101
230042992817,9cyclictest0-21swapper/119:14:041
23004299281,23cyclictest534983-21apt-get00:18:551
23004299281,22cyclictest460616-21apt-get23:03:541
230040992823,4cyclictest316447-21kworker/u8:2+events_unbound20:35:150
230040992823,3cyclictest251597-21unixbench_multi19:29:160
230045992724,2cyclictest435057-21unixbench_singl22:34:132
230045992724,2cyclictest435057-21unixbench_singl22:34:132
230045992724,2cyclictest301512-21unixbench_multi20:19:152
230045992723,3cyclictest215362-21kworker/u8:1+events_unbound21:15:002
230045992723,3cyclictest215362-21kworker/u8:1+events_unbound21:15:002
230045992723,2cyclictest327051-21cpuspeed_turbos20:48:572
230045992721,4cyclictest539361-21kworker/u8:1+events_unbound00:30:022
23004299279,15cyclictest520003-21sshd00:01:421
23004299271,22cyclictest470114-21runrttasks23:10:021
23004299271,21cyclictest430799-21apt-get22:33:531
23004299271,21cyclictest430799-21apt-get22:33:531
230040992723,3cyclictest215362-21kworker/u8:1+flush-8:019:32:470
230040992723,2cyclictest355849-21idleruntime-cro21:15:010
230040992723,2cyclictest355849-21idleruntime-cro21:15:000
230040992722,4cyclictest487982-21kthreadcore23:29:020
230040992719,6cyclictest316425-21latency_hist20:35:010
23004299261,21cyclictest302237-21apt-get20:23:531
230040992624,1cyclictest0-21swapper/020:04:100
230040992622,3cyclictest215362-21kworker/u8:1+flush-8:021:53:580
230040992620,4cyclictest268226-21sort19:49:020
2252502262,5sleep00-21swapper/019:06:550
23004599250,23cyclictest262686-21tr19:43:592
230040992520,4cyclictest444948-21kworker/u8:3+events_unbound22:45:200
230040992518,3cyclictest0-21swapper/000:18:560
230045992421,2cyclictest321329-21cron20:40:002
230045992420,2cyclictest558875-21unixbench_multi00:39:132
230045992418,4cyclictest291611-21cat20:09:592
23004299241,19cyclictest230102-21apt-config19:10:021
230040992420,3cyclictest215362-21kworker/u8:1+flush-8:019:39:120
230040992418,4cyclictest237167-21kworker/u8:3+events_unbound19:59:040
23004599236,2cyclictest0-21swapper/200:24:042
23004599232,4cyclictest0-21swapper/222:40:002
230045992320,2cyclictest0-21swapper/221:35:012
230045992319,3cyclictest450195-21kworker/u8:0+flush-8:023:19:082
230045992319,3cyclictest410475-21cron22:10:002
230045992318,3cyclictest517153-21kthreadcore23:59:032
230045992318,3cyclictest514313-21unixbench_multi23:54:142
230045992318,3cyclictest495507-21cpuspeed_turbos23:38:592
230045992317,4cyclictest231105-21irqcore19:14:002
230040992319,3cyclictest499904-21kworker/u8:3+events_unbound23:48:510
230040992319,3cyclictest499904-21kworker/u8:3+events_unbound23:48:510
230040992319,2cyclictest0-21swapper/020:29:120
230040992318,4cyclictest298235-21tr20:19:040
230040992318,3cyclictest215362-21kworker/u8:1+events_unbound19:55:010
230040992317,5cyclictest450195-21kworker/u8:0+events_unbound23:40:000
23004599224,15cyclictest336109-21cron20:55:002
230045992218,2cyclictest530076-21aten2_r5power_c00:13:562
230040992219,2cyclictest425332-21kworker/u8:1+events_unbound22:36:220
230045992118,2cyclictest366715-21cpuspeed_turbos21:28:582
230045992117,3cyclictest272125-21kworker/u8:0+events_unbound19:50:062
230045992117,2cyclictest252319-21aten2_r5power_c19:33:552
230045992115,3cyclictest422728-21kthreadcore22:24:032
23004599211,18cyclictest236967-21tune2fs19:15:042
230040992114,5cyclictest559041-21latency_hist00:40:010
5383712200,2sleep3481ktimers/300:19:063
23004599202,13cyclictest450513-21apt-get22:53:542
230045992016,3cyclictest450195-21kworker/u8:0+events_unbound00:15:022
230045992011,7cyclictest362838-21tr21:24:002
230045992011,7cyclictest278-21systemd-journal20:05:002
23004599199,2cyclictest4599602unixbench_singl22:59:152
23004599193,14cyclictest473644-21tr23:14:052
230045991916,2cyclictest281755-21cron20:00:012
230045991915,2cyclictest0-21swapper/220:43:592
230045991915,2cyclictest0-21swapper/220:23:552
2295152193,8sleep30-21swapper/319:09:153
3810542180,2sleep3471rcuc/321:40:013
23004599188,7cyclictest527261-21tr00:09:032
23004599182,14cyclictest494108-21awk23:34:062
23004599182,12cyclictest340472-21ntpq20:59:102
230045991815,2cyclictest505378-21cpuspeed_turbos23:48:592
230045991815,2cyclictest505378-21cpuspeed_turbos23:48:592
230045991815,2cyclictest0-21swapper/223:09:112
23004599181,4cyclictest0-21swapper/223:24:012
230045991814,2cyclictest247456-21aten2_r5power_e19:28:572
23004599172,5cyclictest0-21swapper/222:46:392
23004599172,2cyclictest4191611kthreadcore22:19:072
230045991714,2cyclictest460835-21cpuspeed_turbos23:03:562
230045991714,2cyclictest345214-21tail21:04:062
230045991714,2cyclictest246857-21idleruntime-cro19:25:002
230045991713,2cyclictest0-21swapper/200:03:572
230045991712,3cyclictest393948-21kthreadcore21:54:032
5144312167,5sleep30-21swapper/323:55:003
23004599162,2cyclictest0-21swapper/200:33:272
23004599162,13cyclictest0-21swapper/223:27:592
23004599162,13cyclictest0-21swapper/222:26:562
23004599162,13cyclictest0-21swapper/219:45:302
230045991614,1cyclictest0-21swapper/222:14:102
230045991613,2cyclictest0-21swapper/221:40:062
230045991612,2cyclictest0-21swapper/221:39:142
23004599161,11cyclictest386429-21apt-get21:48:542
5497412157,5sleep30-21swapper/300:33:563
4548642142,7sleep30-21swapper/322:55:013
4357172133,6sleep30-21swapper/322:38:563
4345172132,7sleep30-21swapper/322:34:073
4345172132,7sleep30-21swapper/322:34:073
4209532132,7sleep30-21swapper/322:23:563
4148192133,6sleep30-21swapper/322:14:093
3907802132,6sleep30-21swapper/321:50:013
3016572132,6sleep30-21swapper/320:20:013
5442632122,6sleep30-21swapper/300:25:013
5403442122,6sleep30-21swapper/300:24:013
5155052122,6sleep30-21swapper/323:59:013
4957572122,6sleep30-21swapper/323:39:033
4760642122,6sleep30-21swapper/323:19:013
4704422122,7sleep30-21swapper/323:13:553
4263642122,6sleep30-21swapper/322:29:013
4154282122,6sleep30-21swapper/322:15:013
4103132122,6sleep30-21swapper/322:09:153
3960162122,6sleep30-21swapper/321:56:113
3669702122,6sleep30-21swapper/321:29:033
3618152122,6sleep30-21swapper/321:23:573
3561522122,6sleep30-21swapper/321:16:113
3561522122,6sleep30-21swapper/321:16:113
3415732122,6sleep30-21swapper/321:03:553
3262582122,5sleep30-21swapper/320:45:013
3223182122,6sleep30-21swapper/320:44:013
2866842122,5sleep30-21swapper/320:05:003
2781242122,6sleep30-21swapper/319:59:033
2526282122,6sleep30-21swapper/319:33:593
2510742122,6sleep30-21swapper/319:29:093
2421232122,6sleep30-21swapper/319:20:113
2311572122,6sleep30-21swapper/319:14:023
5544282112,6sleep30-21swapper/300:36:053
5305182112,5sleep30-21swapper/300:14:023
5248182112,5sleep30-21swapper/300:05:113
5201722112,5sleep30-21swapper/300:03:553
5048902112,6sleep30-21swapper/323:45:543
5048902112,6sleep30-21swapper/323:45:533
4998982112,6sleep30-21swapper/323:40:273
4901442111,6sleep30-21swapper/323:31:573
4851152112,5sleep30-21swapper/323:25:113
4796802110,6sleep30-21swapper/323:20:013
4653462112,5sleep30-21swapper/323:06:533
4606212112,5sleep30-21swapper/323:03:563
4561882112,6sleep30-21swapper/322:59:023
4454002112,5sleep30-21swapper/322:47:113
4401082112,5sleep30-21swapper/322:40:013
4009882112,5sleep30-21swapper/322:01:413
3915902112,5sleep30-21swapper/321:53:583
3814972112,5sleep30-21swapper/321:43:563
3712122113,5sleep30-21swapper/321:31:393
3508812112,5sleep30-21swapper/321:10:013
3508812112,5sleep30-21swapper/321:10:013
3385502111,6sleep30-21swapper/320:59:053
3326832112,5sleep30-21swapper/320:54:043
3174882112,6sleep30-21swapper/320:39:033
3120432115,4sleep30-21swapper/320:33:543
3025472112,5sleep30-21swapper/320:23:583
2927712112,5sleep30-21swapper/320:14:033
2864562112,5sleep30-21swapper/320:04:123
2723462112,6sleep30-21swapper/319:52:573
2716482110,7sleep30-21swapper/319:50:003
2615722112,5sleep30-21swapper/319:40:013
2569312113,5sleep30-21swapper/319:35:273
2374572112,5sleep30-21swapper/319:18:543
3068022102,5sleep30-21swapper/320:25:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional