You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 12:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Nov 30, 2025 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
561323991321,4cyclictest0-21swapper/000:00:020
56133099777,45cyclictest291rcuc/219:55:032
561323995521,15cyclictest131rcu_preempt21:48:310
561326995440,8cyclictest561333-21grep19:10:011
561326995428,17cyclictest272-21dbus-daemon21:40:011
561326995428,17cyclictest272-21dbus-daemon21:40:001
56132699524,7cyclictest23-21ksoftirqd/122:50:011
56133599514,6cyclictest131rcu_preempt00:15:013
561326995119,20cyclictest23-21ksoftirqd/123:25:021
56132399513,30cyclictest151rcuc/019:55:020
56133599504,6cyclictest131rcu_preempt00:30:013
56132399501,7cyclictest131rcu_preempt20:15:000
56133599492,5cyclictest131rcu_preempt19:30:023
56133099491,7cyclictest131rcu_preempt00:00:022
561323994934,11cyclictest272-21dbus-daemon19:20:000
561323994934,11cyclictest235-21systemd-journal20:21:300
561323994929,14cyclictest235-21systemd-journal19:34:590
561323994922,14cyclictest12-21ksoftirqd/000:10:010
56132399491,2cyclictest0-21swapper/022:15:020
561323994911,34cyclictest12-21ksoftirqd/020:50:020
56133599483,5cyclictest131rcu_preempt23:40:013
561335994831,13cyclictest272-21dbus-daemon20:25:013
561335994824,21cyclictest37-21ksoftirqd/321:45:013
56133599481,5cyclictest131rcu_preempt21:40:013
56133599481,5cyclictest131rcu_preempt21:40:013
56133099481,3cyclictest131rcu_preempt23:40:002
56132699486,6cyclictest131rcu_preempt22:10:031
561326994831,13cyclictest272-21dbus-daemon22:55:011
56132699481,5cyclictest131rcu_preempt20:01:001
561323994830,13cyclictest235-21systemd-journal23:00:000
56133599474,6cyclictest570064-21latency_hist21:55:033
56133599474,33cyclictest361rcuc/323:30:023
56133599471,5cyclictest131rcu_preempt22:45:023
56133599471,5cyclictest131rcu_preempt20:40:003
561330994733,10cyclictest1-21systemd21:10:002
56132699479,4cyclictest131rcu_preempt19:35:011
561326994736,7cyclictest23-21ksoftirqd/100:10:021
561326994734,7cyclictest567305-21munin-plugin-st21:05:001
56132399476,30cyclictest19050irq/34-mmc119:45:020
56132399471,42cyclictest5740371sshd23:13:260
56133599469,8cyclictest37-21ksoftirqd/321:10:013
56133599467,6cyclictest131rcu_preempt23:10:033
561335994627,14cyclictest235-21systemd-journal22:40:013
56133599462,5cyclictest131rcu_preempt22:00:013
561335994622,7cyclictest575900-21kworker/3:200:00:113
56133599461,5cyclictest131rcu_preempt20:05:013
561335994615,23cyclictest272-21dbus-daemon00:10:013
561326994633,8cyclictest567561-21cron21:10:001
561326994630,11cyclictest272-21dbus-daemon00:00:001
561326994628,13cyclictest272-21dbus-daemon23:40:011
561326994627,14cyclictest272-21dbus-daemon00:05:011
561326994625,12cyclictest306-21in:imuxsock19:40:011
561326994625,12cyclictest306-21in:imuxsock19:40:001
561326994624,15cyclictest568094-21run-parts21:20:001
561326994614,20cyclictest131rcu_preempt19:30:001
56132399462,3cyclictest131rcu_preempt22:40:010
56132399462,3cyclictest131rcu_preempt00:31:550
56132399462,3cyclictest131rcu_preempt00:31:550
561323994623,4cyclictest12-21ksoftirqd/023:30:020
56132399462,13cyclictest570174-21sendmail22:00:010
56133599454,5cyclictest131rcu_preempt20:20:033
561335994537,5cyclictest37-21ksoftirqd/323:45:023
561335994529,11cyclictest37-21ksoftirqd/300:00:003
561335994520,20cyclictest37-21ksoftirqd/323:32:313
561335994520,20cyclictest37-21ksoftirqd/323:32:313
56133599451,5cyclictest131rcu_preempt19:55:003
56133599451,15cyclictest0-21swapper/323:05:013
56133099453,3cyclictest131rcu_preempt23:45:002
561330994532,10cyclictest572556-21sed22:45:012
56133099451,6cyclictest0-21swapper/220:00:002
56133099451,6cyclictest0-21swapper/219:59:592
561326994537,6cyclictest23-21ksoftirqd/121:00:021
561326994537,6cyclictest23-21ksoftirqd/121:00:011
561326994525,9cyclictest306-21in:imuxsock00:40:011
561326994524,9cyclictest306-21in:imuxsock21:15:001
561326994515,6cyclictest131rcu_preempt22:15:001
56132399455,6cyclictest131rcu_preempt20:40:010
561323994529,13cyclictest575327-21sshd23:36:310
561335994428,12cyclictest272-21dbus-daemon21:00:013
561335994428,12cyclictest272-21dbus-daemon21:00:013
561335994423,5cyclictest37-21ksoftirqd/322:10:013
561335994421,19cyclictest37-21ksoftirqd/323:15:013
561335994420,5cyclictest37-21ksoftirqd/323:00:013
561335994419,20cyclictest361rcuc/319:20:023
561330994435,5cyclictest0-21swapper/219:10:022
56132699447,30cyclictest575108-21cut23:35:011
56132699447,30cyclictest575108-21cut23:35:011
561326994431,7cyclictest306-21in:imuxsock20:35:021
561326994425,13cyclictest1-21systemd22:30:011
561326994414,7cyclictest131rcu_preempt19:45:001
561326994414,6cyclictest131rcu_preempt19:15:021
561326994413,6cyclictest131rcu_preempt20:35:011
561323994429,11cyclictest0-21swapper/000:00:060
561323994427,13cyclictest1-21systemd22:45:010
561323994422,15cyclictest235-21systemd-journal22:04:590
561323994420,19cyclictest12-21ksoftirqd/022:30:000
56132399441,3cyclictest131rcu_preempt00:40:000
56133599438,6cyclictest131rcu_preempt21:46:223
56133599434,7cyclictest37-21ksoftirqd/319:15:033
56133599433,5cyclictest131rcu_preempt22:20:313
561335994326,13cyclictest37-21ksoftirqd/321:35:013
561335994321,6cyclictest37-21ksoftirqd/320:34:593
561335994320,19cyclictest37-21ksoftirqd/320:55:013
561335994319,4cyclictest37-21ksoftirqd/320:00:003
561335994319,4cyclictest37-21ksoftirqd/319:59:593
561335994318,5cyclictest37-21ksoftirqd/319:50:013
56133099434,5cyclictest131rcu_preempt20:30:012
56133099434,5cyclictest131rcu_preempt20:30:012
561330994333,8cyclictest30-21ksoftirqd/220:45:032
561330994333,8cyclictest30-21ksoftirqd/220:45:032
561330994318,21cyclictest30-21ksoftirqd/220:20:002
561330994315,7cyclictest131rcu_preempt19:35:012
56133099431,3cyclictest131rcu_preempt22:10:022
561326994329,10cyclictest23-21ksoftirqd/123:55:011
561326994327,12cyclictest272-21dbus-daemon21:35:011
561326994326,10cyclictest306-21in:imuxsock19:55:011
561326994321,17cyclictest574329-21run-parts23:20:011
561326994321,17cyclictest574329-21run-parts23:20:011
561323994330,9cyclictest12-21ksoftirqd/021:20:010
56132399432,37cyclictest561376-21ls19:10:010
561323994315,4cyclictest12-21ksoftirqd/000:30:020
561323994313,22cyclictest12-21ksoftirqd/020:16:030
56133599427,6cyclictest131rcu_preempt00:34:413
56133599427,6cyclictest131rcu_preempt00:34:413
561335994220,5cyclictest37-21ksoftirqd/321:15:013
561335994220,4cyclictest37-21ksoftirqd/320:15:013
561335994220,19cyclictest37-21ksoftirqd/322:50:013
561335994219,5cyclictest37-21ksoftirqd/322:10:033
561335994219,5cyclictest37-21ksoftirqd/320:50:013
561335994219,5cyclictest37-21ksoftirqd/300:40:023
561335994217,5cyclictest37-21ksoftirqd/323:25:003
56133599421,5cyclictest131rcu_preempt22:20:013
56133099423,3cyclictest131rcu_preempt22:59:592
561330994229,10cyclictest1-21systemd21:15:012
561330994227,10cyclictest567014-21cron21:00:012
561330994227,10cyclictest567014-21cron21:00:002
561330994223,15cyclictest1-21systemd23:30:012
561330994220,19cyclictest30-21ksoftirqd/222:38:312
561326994231,6cyclictest577213-21latency_hist00:15:011
561326994225,11cyclictest235-21systemd-journal20:30:011
561326994225,11cyclictest235-21systemd-journal20:30:001
56132699421,5cyclictest131rcu_preempt22:45:021
561326994212,6cyclictest131rcu_preempt23:05:001
56132399428,5cyclictest131rcu_preempt19:25:020
56132399428,5cyclictest131rcu_preempt19:25:010
561323994230,8cyclictest272-21dbus-daemon22:25:020
56132399422,7cyclictest131rcu_preempt22:10:020
561323994227,12cyclictest566691-21latency_hist20:50:020
56132399422,3cyclictest131rcu_preempt21:25:020
56132399422,36cyclictest568853-21modprobe21:34:310
56132399421,3cyclictest131rcu_preempt23:16:320
56132399421,3cyclictest131rcu_preempt23:16:320
56132399421,3cyclictest131rcu_preempt20:10:020
561323994210,24cyclictest0-21swapper/019:40:010
561323994210,24cyclictest0-21swapper/019:40:000
561335994121,4cyclictest37-21ksoftirqd/300:20:013
561335994120,3cyclictest37-21ksoftirqd/320:10:003
561335994119,5cyclictest37-21ksoftirqd/320:30:023
561335994119,5cyclictest37-21ksoftirqd/320:30:023
561335994119,18cyclictest37-21ksoftirqd/320:45:013
561335994119,18cyclictest37-21ksoftirqd/320:45:003
561335994117,5cyclictest37-21ksoftirqd/323:55:003
561335994117,4cyclictest37-21ksoftirqd/321:05:003
56133599411,5cyclictest131rcu_preempt23:50:023
56133599411,5cyclictest131rcu_preempt23:50:013
56133099414,3cyclictest131rcu_preempt20:05:002
56133099413,33cyclictest574371-21grep23:20:022
56133099413,33cyclictest574371-21grep23:20:012
561330994130,8cyclictest568085-21cron21:20:002
561330994129,9cyclictest577487-21systemd-run00:20:012
561330994127,11cyclictest30-21ksoftirqd/200:01:492
561330994125,11cyclictest272-21dbus-daemon21:55:002
56133099411,6cyclictest0-21swapper/221:45:022
561330994115,3cyclictest150-21pr/ttyS200:10:022
561326994131,8cyclictest23-21ksoftirqd/122:00:001
561326994129,9cyclictest561772-21cron19:19:591
561326994129,8cyclictest23-21ksoftirqd/122:25:001
561326994127,9cyclictest571332-21grep22:20:021
561326994114,20cyclictest272-21dbus-daemon23:25:001
561326994110,6cyclictest131rcu_preempt00:25:011
561323994135,4cyclictest16950irq/224-800000023:05:010
56132399413,4cyclictest131rcu_preempt21:10:310
56132399413,3cyclictest131rcu_preempt00:15:010
561323994128,10cyclictest12-21ksoftirqd/023:25:010
56132399412,3cyclictest131rcu_preempt19:30:000
561323994115,5cyclictest12-21ksoftirqd/021:10:010
561323994114,9cyclictest561705-21ls19:15:030
56132399411,3cyclictest131rcu_preempt21:40:000
56132399411,3cyclictest131rcu_preempt21:39:590
56132399411,3cyclictest131rcu_preempt00:20:000
56133599404,32cyclictest571721-21cron22:29:593
56133599402,6cyclictest131rcu_preempt22:50:023
561335994025,11cyclictest272-21dbus-daemon23:20:013
561335994025,11cyclictest272-21dbus-daemon23:20:003
561335994024,12cyclictest1-21systemd19:40:013
561335994024,12cyclictest1-21systemd19:40:003
561335994020,5cyclictest37-21ksoftirqd/319:35:013
561335994019,5cyclictest37-21ksoftirqd/300:25:003
561335994019,17cyclictest37-21ksoftirqd/321:30:003
561335994018,5cyclictest37-21ksoftirqd/321:25:013
56133099404,3cyclictest131rcu_preempt22:35:002
56133099403,3cyclictest131rcu_preempt23:25:002
561330994029,7cyclictest30-21ksoftirqd/220:39:592
561330994028,9cyclictest571742-21sh22:30:012
561330994028,9cyclictest561608-21cut19:15:012
561330994028,8cyclictest564801-21sed20:15:022
561330994024,11cyclictest272-21dbus-daemon22:50:012
56133099401,3cyclictest131rcu_preempt21:30:002
56132699404,8cyclictest23-21ksoftirqd/121:25:011
561326994018,17cyclictest272-21dbus-daemon22:05:011
561326994018,16cyclictest235-21systemd-journal21:50:011
56132699401,5cyclictest131rcu_preempt22:35:001
56132699401,4cyclictest131rcu_preempt19:25:021
56132699401,4cyclictest131rcu_preempt19:25:021
561326994014,6cyclictest131rcu_preempt22:40:011
561326994013,7cyclictest131rcu_preempt20:20:001
561326994012,21cyclictest573275-21modprobe23:00:001
561326994010,6cyclictest131rcu_preempt20:15:011
56132399403,3cyclictest131rcu_preempt23:50:030
56132399403,3cyclictest131rcu_preempt23:50:020
56132399403,3cyclictest131rcu_preempt20:01:000
56132399401,6cyclictest131rcu_preempt23:10:020
561323994014,4cyclictest12-21ksoftirqd/000:25:000
56132399401,3cyclictest131rcu_preempt21:55:030
56132399401,3cyclictest131rcu_preempt21:00:010
56132399401,3cyclictest131rcu_preempt21:00:010
56133599394,5cyclictest131rcu_preempt22:35:023
561335993929,7cyclictest570419-21sshd22:04:313
561335993917,5cyclictest37-21ksoftirqd/319:45:003
561335993914,6cyclictest131rcu_preempt19:10:023
561330993932,5cyclictest30-21ksoftirqd/223:35:022
561330993932,5cyclictest30-21ksoftirqd/223:35:022
561330993929,7cyclictest569745-21cut21:50:032
561330993929,7cyclictest562291-21sshd19:27:272
56133099392,3cyclictest131rcu_preempt21:25:012
561330993916,4cyclictest30-21ksoftirqd/222:55:022
56133099391,3cyclictest131rcu_preempt00:15:012
56133099391,35cyclictest30-21ksoftirqd/223:55:012
56132699397,25cyclictest563356-21grep19:49:591
56132699395,29cyclictest569402-21munin-plugin-st21:45:001
561326993931,5cyclictest23-21ksoftirqd/120:10:001
561326993927,9cyclictest23-21ksoftirqd/100:35:031
561326993927,9cyclictest23-21ksoftirqd/100:35:021
56132399393,3cyclictest131rcu_preempt22:50:020
56132399392,3cyclictest131rcu_preempt20:41:080
56132399392,3cyclictest131rcu_preempt20:41:070
561323993923,12cyclictest272-21dbus-daemon23:55:000
561323993915,6cyclictest12-21ksoftirqd/020:00:000
561323993915,6cyclictest12-21ksoftirqd/020:00:000
56132399391,3cyclictest131rcu_preempt20:35:000
56132399391,3cyclictest131rcu_preempt20:30:040
56132399391,3cyclictest131rcu_preempt20:30:030
561335993821,10cyclictest37-21ksoftirqd/321:18:323
56133099384,3cyclictest131rcu_preempt21:38:062
56133099384,3cyclictest131rcu_preempt21:38:062
56133099383,3cyclictest131rcu_preempt19:50:002
561330993828,8cyclictest30-21ksoftirqd/222:25:012
561330993826,8cyclictest561793-21cat19:20:002
561330993825,10cyclictest1-21systemd23:05:012
561330993822,12cyclictest574001-21sed23:10:032
561330993816,5cyclictest30-21ksoftirqd/219:25:012
561330993816,5cyclictest30-21ksoftirqd/219:25:002
56133099381,5cyclictest131rcu_preempt20:10:022
56133099381,5cyclictest0-21swapper/220:50:012
56133099381,3cyclictest131rcu_preempt00:35:032
56133099381,3cyclictest131rcu_preempt00:35:022
56132699388,6cyclictest131rcu_preempt23:11:321
56132699388,5cyclictest131rcu_preempt23:09:321
56132699388,17cyclictest577443-21modprobe00:20:001
56132699387,9cyclictest23-21ksoftirqd/121:26:311
56132699387,26cyclictest235-21systemd-journal00:30:001
56132699383,25cyclictest565292-21latency_hist20:25:011
56132699381,4cyclictest131rcu_preempt23:45:011
561323993829,5cyclictest12-21ksoftirqd/019:50:010
561323993826,9cyclictest235-21systemd-journal22:33:560
561323993822,12cyclictest575607-21sshd23:41:320
56132399381,3cyclictest131rcu_preempt21:29:590
561323993811,22cyclictest0-21swapper/022:53:420
56133599372,5cyclictest131rcu_preempt19:20:273
56133599372,5cyclictest131rcu_preempt19:20:263
561330993727,7cyclictest578140-21cut00:30:022
56133099372,3cyclictest131rcu_preempt22:00:022
561330993719,3cyclictest30-21ksoftirqd/200:40:012
56133099371,5cyclictest131rcu_preempt22:20:002
56133099371,3cyclictest131rcu_preempt21:05:022
56132699377,7cyclictest23-21ksoftirqd/120:00:011
56132699377,7cyclictest23-21ksoftirqd/120:00:011
56132699376,7cyclictest131rcu_preempt20:45:011
56132699376,7cyclictest131rcu_preempt20:45:001
56132699375,20cyclictest575890-21munin-plugin-st23:50:001
56132699375,20cyclictest575890-21munin-plugin-st23:50:001
56132699373,10cyclictest23-21ksoftirqd/121:55:011
561323993716,18cyclictest16850irq/134-800000022:13:310
561330993627,5cyclictest30-21ksoftirqd/200:21:532
561330993619,9cyclictest571119-21grep22:15:022
56133099361,5cyclictest131rcu_preempt21:35:012
56133099361,3cyclictest131rcu_preempt23:11:382
56133099361,3cyclictest131rcu_preempt19:45:012
56132399368,6cyclictest12-21ksoftirqd/023:34:200
56132399368,6cyclictest12-21ksoftirqd/023:34:200
56133099354,3cyclictest131rcu_preempt19:36:222
56133099354,3cyclictest131rcu_preempt19:36:222
561330993527,5cyclictest565228-21latency_hist20:25:022
561330993527,5cyclictest306-21in:imuxsock20:52:312
561326993523,8cyclictest23-21ksoftirqd/120:50:011
56132699351,10cyclictest566774-21munin-plugin-st20:55:001
56132399353,5cyclictest131rcu_preempt21:43:060
56132399351,3cyclictest131rcu_preempt21:05:000
56133099342,3cyclictest131rcu_preempt23:50:012
56133099342,3cyclictest131rcu_preempt23:50:002
56133099332,3cyclictest131rcu_preempt22:01:252
561330993212,3cyclictest30-21ksoftirqd/220:32:302
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional