You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 03:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Fri Dec 05, 2025 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1176521997935,13cyclictest272-21dbus-daemon21:40:011
1176521997935,13cyclictest272-21dbus-daemon21:40:001
117651799787,31cyclictest0-21swapper/019:30:020
1176521997630,5cyclictest306-21in:imuxsock19:50:011
1176517997513,31cyclictest1-21systemd00:01:270
1176517997513,31cyclictest1-21systemd00:01:270
117652199724,4cyclictest131rcu_preempt20:00:031
117652199724,4cyclictest131rcu_preempt20:00:021
1176521997231,10cyclictest235-21systemd-journal21:42:271
1176526997024,12cyclictest235-21systemd-journal21:10:012
1176521996925,17cyclictest272-21dbus-daemon19:15:001
1176517996933,31cyclictest1192554-21sshd00:18:140
1176530996820,17cyclictest1179661-21latency_hist20:10:013
1176530996416,10cyclictest37-21ksoftirqd/323:40:013
1176530996416,10cyclictest37-21ksoftirqd/323:40:003
117651799639,14cyclictest1-21systemd20:45:000
1176517996319,15cyclictest1187091-21sshd22:30:270
1176517996318,13cyclictest1179269-21date20:00:030
1176517996318,13cyclictest1179269-21date20:00:020
1176517996124,4cyclictest1190784-21munin-run23:45:010
1176521996040,14cyclictest1-21systemd20:35:001
1176517996022,34cyclictest363-21snmpd19:35:160
1176517995919,36cyclictest0-21swapper/020:28:260
1176517995737,16cyclictest1189438-21sshd23:18:270
1176517995712,22cyclictest1176511-21cyclictest20:05:010
1176517995618,34cyclictest1179817-21sshd20:13:260
1176517995618,34cyclictest1179817-21sshd20:13:250
1176517995615,36cyclictest363-21snmpd23:46:050
1176517995526,6cyclictest1187385-21latency_hist22:40:020
1176530995434,14cyclictest272-21dbus-daemon20:15:023
1176530995434,14cyclictest272-21dbus-daemon20:15:013
1176517995437,14cyclictest1177232-21sshd19:22:250
1176517995433,17cyclictest1185812-21sshd22:09:260
1176517995420,11cyclictest1176616-21latency_hist19:10:020
1176530995338,11cyclictest1180845-21sshd20:30:253
1176530995320,5cyclictest1177512-21munin-run19:30:003
1176530995311,11cyclictest1192643-21sed00:20:023
1176517995335,14cyclictest1189164-21sshd23:11:270
1176517995317,32cyclictest0-21swapper/023:40:010
1176517995317,32cyclictest0-21swapper/023:40:000
1176530995237,11cyclictest1184514-21sshd21:42:453
1176517995236,13cyclictest1186929-21cat22:30:030
1176517995236,12cyclictest1186060-21sshd22:13:280
1176530995137,11cyclictest1177746-21sshd19:34:263
1176521995134,13cyclictest272-21dbus-daemon19:40:001
1176521995113,27cyclictest1182162-21modprobe21:00:001
1176517995136,11cyclictest1182929-21sshd21:10:270
1176517995135,13cyclictest1178515-21sshd19:46:260
1176530995036,10cyclictest1183192-21cron21:16:593
1176530995035,11cyclictest1191824-21munin-run00:05:023
1176530995035,11cyclictest1191824-21munin-run00:05:023
1176530995035,11cyclictest1186385-21munin-run22:20:013
1176530995034,12cyclictest1188377-21sshd22:55:263
1176530995034,12cyclictest1184768-21sshd21:47:453
1176530995034,11cyclictest1-21systemd22:15:013
1176530995034,11cyclictest1177021-21munin-run19:20:013
1176530995032,14cyclictest1193077-21sshd00:26:283
1176517995037,10cyclictest1183981-21sshd21:32:280
1176517995035,12cyclictest1182428-21sshd21:02:380
1176517995035,11cyclictest1181887-21sshd20:51:250
1176530994935,11cyclictest1189467-21munin-run23:20:013
1176530994933,12cyclictest1188126-21sshd22:50:283
1176530994931,15cyclictest1178510-21sshd19:45:263
1176526994910,27cyclictest1189184-21kworker/u8:023:25:022
1176521994930,14cyclictest272-21dbus-daemon00:10:011
1176517994936,10cyclictest363-21snmpd23:21:000
1176517994935,10cyclictest1184302-21munin-run21:40:020
1176517994935,10cyclictest1184302-21munin-run21:40:010
1176517994913,31cyclictest0-21swapper/021:55:270
117653099488,6cyclictest131rcu_preempt00:15:003
117653099485,6cyclictest131rcu_preempt19:45:013
1176530994834,10cyclictest1180392-21munin-run20:25:013
1176530994834,10cyclictest1180392-21munin-run20:25:013
1176530994833,12cyclictest1192027-21sshd00:05:263
1176530994833,12cyclictest1191021-21munin-run23:49:593
1176530994833,11cyclictest1178799-21munin-run19:55:003
1176530994832,12cyclictest1189171-21sshd23:12:593
117652699485,6cyclictest131rcu_preempt20:32:272
1176526994826,16cyclictest1-21systemd20:15:022
1176526994826,16cyclictest1-21systemd20:15:022
1176521994833,10cyclictest23-21ksoftirqd/122:39:591
1176517994834,11cyclictest1192045-21sshd00:09:270
1176517994833,12cyclictest1187884-21sshd22:47:270
1176517994833,12cyclictest1183479-21sshd21:22:270
1176517994833,12cyclictest1183479-21sshd21:22:260
1176517994833,11cyclictest1179584-21munin-run20:10:000
117651799482,14cyclictest1180310-21latency_hist20:20:020
1176517994813,30cyclictest1191558-21lockfile-remove00:00:000
1176530994734,10cyclictest363-21snmpd19:55:203
1176530994734,10cyclictest363-21snmpd19:55:193
1176530994734,10cyclictest1191286-21munin-run23:55:013
1176530994734,10cyclictest1184230-21sshd21:36:263
1176530994734,10cyclictest1184230-21sshd21:36:263
1176530994734,10cyclictest1176688-21sshd19:12:233
1176530994733,10cyclictest1185257-21sshd21:55:273
1176530994733,10cyclictest1182942-21sshd21:12:403
1176530994732,9cyclictest37-21ksoftirqd/322:40:023
1176530994732,11cyclictest1188910-21sshd23:05:263
117652699477,3cyclictest131rcu_preempt21:50:012
1176526994731,12cyclictest30-21ksoftirqd/200:30:012
117652699472,4cyclictest131rcu_preempt23:40:012
117652699472,4cyclictest131rcu_preempt23:40:002
1176517994739,5cyclictest16950irq/224-800000021:20:000
1176517994735,9cyclictest1182666-21sshd21:05:270
1176517994734,10cyclictest1181100-21sshd20:35:270
1176517994734,10cyclictest1181100-21sshd20:35:260
1176517994734,10cyclictest1176694-21sshd19:13:240
1176517994733,11cyclictest1188246-21cat22:55:020
1176517994733,10cyclictest363-21snmpd23:05:580
1176517994732,12cyclictest1185025-21sshd21:54:280
1176517994732,12cyclictest1185025-21sshd21:54:270
1176517994732,11cyclictest363-21snmpd00:37:280
1176517994732,10cyclictest1177021-21munin-run19:20:010
1176517994729,13cyclictest272-21dbus-daemon00:35:010
117651799472,40cyclictest1181642-21sshd20:47:330
1176530994638,4cyclictest1183725-21sshd21:26:263
1176530994633,10cyclictest363-21snmpd20:00:193
1176530994632,11cyclictest1181393-21sshd20:43:263
1176530994632,10cyclictest1182451-21munin-run21:05:013
1176530994631,11cyclictest1185016-21sshd21:52:283
1176530994631,11cyclictest1185016-21sshd21:52:283
1176530994631,11cyclictest1183507-21munin-run21:25:013
1176530994631,11cyclictest1183507-21munin-run21:25:003
1176530994630,11cyclictest1180109-21munin-run20:20:013
1176530994624,17cyclictest306-21in:imuxsock22:10:003
1176530994624,16cyclictest1190228-21cron23:35:003
117652699467,33cyclictest1182453-21cut21:05:012
1176526994631,12cyclictest235-21systemd-journal22:09:262
1176526994618,24cyclictest30-21ksoftirqd/223:00:022
1176521994629,12cyclictest235-21systemd-journal00:00:011
1176517994634,8cyclictest0-21swapper/021:43:390
1176517994633,9cyclictest363-21snmpd23:04:440
1176517994633,10cyclictest1192292-21sshd00:13:130
1176517994632,11cyclictest1184772-21sshd21:48:260
1176517994632,10cyclictest48-21kcompactd022:02:080
1176517994632,10cyclictest48-21kcompactd022:02:070
1176517994629,13cyclictest363-21snmpd19:50:430
1176517994627,14cyclictest363-21snmpd23:52:100
1176530994533,9cyclictest1189973-21sshd23:29:273
1176530994533,8cyclictest1190748-21sshd23:43:053
1176530994532,10cyclictest363-21snmpd22:20:493
1176530994532,10cyclictest1182148-21sshd20:57:373
1176530994531,9cyclictest1181919-21munin-run20:55:003
1176530994531,11cyclictest1189719-21sshd23:24:273
1176530994531,11cyclictest1187636-21sshd22:42:543
1176530994531,11cyclictest1186839-21sshd22:25:273
1176530994524,14cyclictest1193332-21sshd00:32:283
1176526994525,13cyclictest272-21dbus-daemon20:25:012
1176526994525,13cyclictest272-21dbus-daemon20:25:002
117652199457,23cyclictest1192309-21modprobe00:14:591
117652199455,6cyclictest23-21ksoftirqd/100:29:281
1176521994530,10cyclictest235-21systemd-journal20:20:001
1176517994536,5cyclictest1188416-21sh23:00:000
1176517994536,5cyclictest1177736-21sshd19:32:250
1176517994532,9cyclictest48-21kcompactd021:26:290
1176517994532,9cyclictest363-21snmpd00:24:290
1176517994531,10cyclictest1180378-21munin-run20:25:000
1176517994531,10cyclictest1180378-21munin-run20:25:000
1176517994522,18cyclictest1-21systemd00:30:000
1176517994519,22cyclictest16850irq/134-800000022:20:000
1176530994432,9cyclictest1192833-21sshd00:23:143
1176530994432,9cyclictest1191489-21sshd23:55:263
1176530994432,9cyclictest1187874-21sshd22:45:273
1176526994433,8cyclictest1189180-21sshd23:14:272
1176521994426,14cyclictest1-21systemd20:55:001
117652199441,3cyclictest131rcu_preempt23:44:591
1176521994412,5cyclictest131rcu_preempt20:45:291
1176517994434,7cyclictest1190213-21sshd23:32:270
1176517994434,7cyclictest1186587-21sshd22:20:270
1176517994430,10cyclictest235-21systemd-journal20:57:270
1176530994332,8cyclictest1187098-21sshd22:32:263
1176530994330,9cyclictest363-21snmpd22:00:363
1176530994330,9cyclictest363-21snmpd22:00:353
1176530994330,10cyclictest1181642-21sshd20:47:333
1176530994329,10cyclictest363-21snmpd19:20:133
1176530994329,10cyclictest1188682-21cron23:05:003
1176530994326,14cyclictest363-21snmpd00:36:153
1176530994324,14cyclictest363-21snmpd21:06:483
1176526994330,10cyclictest1182978-21pluginstate21:15:012
1176526994328,10cyclictest235-21systemd-journal00:09:592
117652199436,29cyclictest1187912-21grep22:50:011
117651799433,6cyclictest131rcu_preempt23:30:000
1176517994336,4cyclictest1178271-21sshd19:42:270
1176517994330,10cyclictest363-21snmpd22:42:580
1176517994327,12cyclictest1180859-21sshd20:32:300
1176530994230,9cyclictest1181176-21munin-run20:40:023
1176530994230,9cyclictest1181176-21munin-run20:40:013
1176530994229,9cyclictest272-21dbus-daemon21:35:013
1176530994224,14cyclictest1180607-21sshd20:27:313
117652699425,20cyclictest30-21ksoftirqd/222:45:022
117652699424,22cyclictest1180096-21modprobe20:19:592
1176526994228,11cyclictest1176538-21cat19:10:012
1176526994222,15cyclictest1179058-21run-parts20:00:002
1176526994222,15cyclictest1179058-21run-parts20:00:002
117652199425,4cyclictest131rcu_preempt19:10:011
1176521994233,6cyclictest23-21ksoftirqd/119:25:011
1176521994229,10cyclictest1180788-21cat20:30:011
1176521994223,14cyclictest235-21systemd-journal00:03:271
1176521994223,14cyclictest235-21systemd-journal00:03:261
1176530994131,8cyclictest48-21kcompactd019:35:523
117652699419,5cyclictest131rcu_preempt22:52:562
117652699416,7cyclictest30-21ksoftirqd/219:20:002
1176526994125,11cyclictest272-21dbus-daemon21:45:002
1176526994122,13cyclictest30-21ksoftirqd/200:25:002
117652199413,34cyclictest1186896-21cat22:30:011
1176521994128,10cyclictest23-21ksoftirqd/119:45:011
1176526994029,8cyclictest1183973-21sshd21:30:272
1176526994027,10cyclictest272-21dbus-daemon21:40:022
1176526994027,10cyclictest272-21dbus-daemon21:40:012
1176526994020,15cyclictest1191530-21cat00:00:012
117652199406,18cyclictest1184793-21grep21:50:001
117652199405,18cyclictest235-21systemd-journal23:19:591
1176521994032,6cyclictest23-21ksoftirqd/122:10:021
1176521994029,7cyclictest1189738-21grep23:25:001
1176521994028,9cyclictest1190470-21sshd23:38:271
1176521994028,9cyclictest1190470-21sshd23:38:271
117652199402,7cyclictest23-21ksoftirqd/120:06:261
1176521994025,12cyclictest272-21dbus-daemon22:55:001
117652699396,3cyclictest131rcu_preempt19:40:012
117652699395,3cyclictest131rcu_preempt20:05:022
117652699394,4cyclictest131rcu_preempt22:35:012
1176526993929,7cyclictest306-21in:imuxsock23:55:012
1176526993926,10cyclictest30-21ksoftirqd/221:20:012
117652699392,3cyclictest131rcu_preempt20:40:022
117652699392,3cyclictest131rcu_preempt20:40:022
1176526993922,13cyclictest30-21ksoftirqd/219:35:022
1176526993920,14cyclictest1-21systemd23:05:002
1176521993929,8cyclictest23-21ksoftirqd/120:25:011
1176521993929,8cyclictest23-21ksoftirqd/120:25:001
1176521993929,7cyclictest23-21ksoftirqd/121:25:381
1176521993927,10cyclictest23-21ksoftirqd/122:35:011
117652199392,4cyclictest131rcu_preempt21:55:011
117652199392,4cyclictest131rcu_preempt21:55:001
1176521993921,13cyclictest272-21dbus-daemon20:05:001
117652699385,4cyclictest131rcu_preempt20:59:592
117652699385,3cyclictest131rcu_preempt20:10:022
117652699385,20cyclictest235-21systemd-journal00:15:002
117652699384,4cyclictest131rcu_preempt21:25:002
117652699384,4cyclictest131rcu_preempt21:25:002
117652699384,23cyclictest1189455-21modprobe23:19:592
1176526993823,11cyclictest1177496-21sshd19:29:262
1176526993821,12cyclictest272-21dbus-daemon19:45:002
1176526993820,13cyclictest272-21dbus-daemon20:55:012
117652199383,4cyclictest131rcu_preempt22:45:011
117652199383,3cyclictest131rcu_preempt23:31:261
1176521993828,7cyclictest23-21ksoftirqd/119:30:141
1176521993828,7cyclictest1188924-21sshd23:08:261
1176521993826,9cyclictest1182822-21sed21:10:021
1176521993818,13cyclictest272-21dbus-daemon00:20:021
117652699374,4cyclictest131rcu_preempt23:25:032
117652699373,4cyclictest131rcu_preempt22:38:282
1176526993727,7cyclictest1193346-21sshd00:34:122
1176526993726,8cyclictest30-21ksoftirqd/219:25:012
1176526993724,9cyclictest1-21systemd20:50:002
1176526993724,9cyclictest1-21systemd20:45:012
1176526993722,12cyclictest1190281-21cat23:35:022
117652699371,4cyclictest131rcu_preempt00:03:112
117652699371,4cyclictest131rcu_preempt00:03:102
117652199374,3cyclictest131rcu_preempt21:25:011
117652199374,3cyclictest131rcu_preempt21:25:001
117652199374,3cyclictest131rcu_preempt20:15:031
117652199374,3cyclictest131rcu_preempt20:15:021
117652199374,26cyclictest1188693-21munin-plugin-st23:05:001
1176521993725,10cyclictest23-21ksoftirqd/121:35:011
117652199372,3cyclictest131rcu_preempt22:55:271
1176521993716,16cyclictest272-21dbus-daemon22:05:011
1176521993716,16cyclictest272-21dbus-daemon22:05:001
1176530993626,7cyclictest37-21ksoftirqd/319:10:033
1176526993628,6cyclictest1193601-21sshd00:37:292
1176526993627,7cyclictest30-21ksoftirqd/223:10:022
117652699362,6cyclictest131rcu_preempt22:00:452
117652699362,6cyclictest131rcu_preempt22:00:442
1176526993625,8cyclictest30-21ksoftirqd/222:30:012
1176526993625,8cyclictest1185380-21date22:00:042
117652699362,4cyclictest131rcu_preempt22:50:012
1176521993626,7cyclictest235-21systemd-journal21:00:011
1176521993626,7cyclictest23-21ksoftirqd/121:15:021
1176521993623,9cyclictest272-21dbus-daemon20:45:011
1176521993618,12cyclictest306-21in:imuxsock21:15:021
117652699353,3cyclictest131rcu_preempt22:12:502
1176526993525,7cyclictest30-21ksoftirqd/219:15:032
1176526993523,8cyclictest1-21systemd21:30:012
1176526993522,10cyclictest30-21ksoftirqd/219:55:032
1176526993521,10cyclictest306-21in:imuxsock00:15:022
117652199353,3cyclictest131rcu_preempt19:50:251
1176521993524,9cyclictest23-21ksoftirqd/122:25:011
117652199352,3cyclictest131rcu_preempt22:20:031
1176521993522,10cyclictest23-21ksoftirqd/119:30:011
1176521993519,11cyclictest235-21systemd-journal00:31:271
1176521993517,12cyclictest0-21swapper/100:40:001
117652699343,3cyclictest131rcu_preempt23:42:272
117652699343,3cyclictest131rcu_preempt22:25:002
1176526993418,11cyclictest235-21systemd-journal23:49:592
117652699341,4cyclictest131rcu_preempt22:20:032
117652199344,4cyclictest131rcu_preempt23:30:001
117652199344,3cyclictest131rcu_preempt22:00:001
117652199343,4cyclictest131rcu_preempt00:25:001
117652199342,3cyclictest131rcu_preempt23:53:091
117652699333,4cyclictest131rcu_preempt19:45:252
1176526993322,9cyclictest30-21ksoftirqd/220:30:002
1176521993321,9cyclictest23-21ksoftirqd/123:15:021
117652199331,4cyclictest131rcu_preempt19:20:021
1176521993221,9cyclictest23-21ksoftirqd/120:40:011
1176521993221,9cyclictest23-21ksoftirqd/120:40:001
1176521993216,10cyclictest235-21systemd-journal22:14:591
1176526993122,5cyclictest1185038-21latency_hist21:55:032
1176526993122,5cyclictest1185038-21latency_hist21:55:022
117652199313,3cyclictest131rcu_preempt23:47:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional