You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 06:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Wed Nov 26, 2025 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
64685998719,21cyclictest77714-21latency_hist23:20:021
64685997016,17cyclictest73066-21cat21:50:011
64685997016,16cyclictest72645-21sed21:40:031
64685996816,15cyclictest76236-21latency_hist22:50:011
64685996811,20cyclictest70852-21sed21:05:021
64685996516,15cyclictest70958-21cut21:10:021
64685996516,15cyclictest70958-21cut21:10:011
64685996515,19cyclictest73316-21cut21:55:011
6468199652,8cyclictest12-21ksoftirqd/023:59:590
64685996316,15cyclictest65142-21munin-run19:20:011
64681996342,16cyclictest235-21systemd-journal21:10:000
64681996342,16cyclictest235-21systemd-journal21:10:000
64685996216,16cyclictest78565-21cat23:35:011
64685996145,11cyclictest1-21systemd23:25:011
64685996145,11cyclictest1-21systemd23:25:011
64685996020,14cyclictest69692-21date20:45:021
64685995737,15cyclictest71428-21munin-run21:20:011
64691995635,18cyclictest37-21ksoftirqd/321:02:183
64685995641,11cyclictest72190-21cron21:35:011
64685995641,11cyclictest72190-21cron21:35:001
64685995638,14cyclictest75550-21sshd22:37:411
64685995638,14cyclictest75550-21sshd22:37:411
64681995610,9cyclictest12-21ksoftirqd/020:50:010
64685995540,11cyclictest66661-21sshd19:45:401
64685995537,13cyclictest79984-21sshd00:01:411
64685995537,13cyclictest74007-21sshd22:09:411
6468199557,6cyclictest131rcu_preempt22:39:420
6468199557,6cyclictest131rcu_preempt22:39:410
64685995438,11cyclictest67997-21munin-run20:15:011
64685995436,14cyclictest68992-21sshd20:30:401
64685995436,14cyclictest68992-21sshd20:30:401
64681995426,5cyclictest131rcu_preempt22:02:330
64685995338,11cyclictest73784-21munin-run22:05:011
64685995337,10cyclictest68255-21munin-run20:20:011
6468199537,9cyclictest12-21ksoftirqd/021:50:010
64681995348,2cyclictest0-21swapper/019:30:020
64681995337,11cyclictest12-21ksoftirqd/019:15:000
6469199523,8cyclictest37-21ksoftirqd/319:35:023
64689995235,13cyclictest235-21systemd-journal21:35:002
64689995235,13cyclictest235-21systemd-journal21:35:002
64685995244,4cyclictest75335-21munin-run22:35:021
64685995236,13cyclictest78159-21sshd23:26:421
64685995233,11cyclictest70299-21cron21:00:001
64685995224,22cyclictest74667-21wc22:20:011
6468199528,6cyclictest131rcu_preempt23:20:010
6468199528,5cyclictest131rcu_preempt19:20:000
64689995129,12cyclictest30-21ksoftirqd/219:20:002
64685995136,10cyclictest77157-21munin-run23:10:021
64685995136,10cyclictest75855-21munin-run22:44:591
64685995135,11cyclictest67492-21munin-run20:05:011
64685995132,14cyclictest363-21snmpd23:04:541
64685995132,13cyclictest78685-21modprobe23:40:001
64685995132,13cyclictest78685-21modprobe23:40:001
64685995131,15cyclictest363-21snmpd22:52:371
6468199516,8cyclictest12-21ksoftirqd/023:50:010
64681995145,3cyclictest0-21swapper/021:45:030
64681995145,3cyclictest0-21swapper/021:45:020
64691995029,14cyclictest235-21systemd-journal00:05:003
6469199502,5cyclictest131rcu_preempt00:20:003
64689995033,12cyclictest235-21systemd-journal23:45:002
64689995013,8cyclictest131rcu_preempt23:00:012
64685995035,12cyclictest66419-21sshd19:42:411
64685995035,10cyclictest76622-21munin-run23:00:011
64685995033,13cyclictest1-21systemd00:00:011
64685995032,10cyclictest66153-21cron19:40:001
64685995023,23cyclictest23-21ksoftirqd/120:40:011
64681995029,14cyclictest1-21systemd23:10:000
64685994942,4cyclictest65409-21munin-run19:25:001
64685994936,9cyclictest66946-21munin-run19:55:001
64685994935,11cyclictest67708-21sshd20:07:401
64685994935,10cyclictest79205-21sshd23:48:421
64685994934,11cyclictest74819-21munin-run22:25:001
64685994933,11cyclictest65926-21munin-run19:35:011
64685994930,16cyclictest363-21snmpd00:12:581
64685994930,15cyclictest363-21snmpd22:27:231
64685994930,14cyclictest80731-21sshd00:15:401
64685994923,19cyclictest81048-21modprobe00:25:001
64681994939,5cyclictest16950irq/224-800000000:10:000
6469199488,6cyclictest131rcu_preempt22:40:013
6469199488,6cyclictest131rcu_preempt22:40:003
6468999485,6cyclictest131rcu_preempt22:14:282
64689994841,5cyclictest30-21ksoftirqd/221:10:032
64689994841,5cyclictest30-21ksoftirqd/221:10:022
64689994833,12cyclictest68258-21systemctl20:20:012
64689994832,12cyclictest1-21systemd23:50:002
64689994831,12cyclictest1-21systemd22:35:002
64685994836,9cyclictest70035-21sshd20:52:141
64685994836,9cyclictest363-21snmpd19:57:091
64685994836,9cyclictest363-21snmpd00:38:031
64685994836,8cyclictest363-21snmpd21:56:441
64685994836,8cyclictest363-21snmpd21:56:431
64685994835,10cyclictest72749-21munin-run21:45:011
64685994835,10cyclictest72749-21munin-run21:45:011
64685994833,10cyclictest48-21kcompactd020:27:231
64685994819,23cyclictest77478-21latency_hist23:15:021
6468199487,11cyclictest12-21ksoftirqd/020:20:010
64681994832,12cyclictest272-21dbus-daemon21:20:000
64681994832,11cyclictest272-21dbus-daemon21:30:000
6469199477,6cyclictest131rcu_preempt21:35:023
6469199477,6cyclictest131rcu_preempt21:35:023
64685994735,9cyclictest64851-21sshd19:14:401
64685994734,9cyclictest48-21kcompactd023:52:061
64685994734,9cyclictest48-21kcompactd023:52:061
64685994734,9cyclictest48-21kcompactd023:42:071
64685994734,9cyclictest48-21kcompactd021:10:401
64685994734,9cyclictest48-21kcompactd019:25:151
64685994733,9cyclictest48-21kcompactd000:26:311
64685994733,10cyclictest363-21snmpd00:34:421
64685994723,17cyclictest71700-21munin-run21:25:001
6468199478,8cyclictest12-21ksoftirqd/019:10:010
6468199477,9cyclictest12-21ksoftirqd/022:45:000
6468199473,13cyclictest12-21ksoftirqd/023:40:010
6468199473,13cyclictest12-21ksoftirqd/023:40:010
6468199471,8cyclictest1-21systemd20:25:000
64681994710,5cyclictest131rcu_preempt22:25:020
6469199467,6cyclictest131rcu_preempt20:15:003
6469199466,8cyclictest37-21ksoftirqd/320:20:403
64691994627,15cyclictest272-21dbus-daemon19:30:003
64691994618,5cyclictest131rcu_preempt23:20:013
6468999461,5cyclictest131rcu_preempt22:40:012
6468999461,5cyclictest131rcu_preempt22:40:012
64685994640,3cyclictest235-21systemd-journal22:14:131
6468199465,8cyclictest12-21ksoftirqd/020:00:020
6468199464,6cyclictest131rcu_preempt00:35:010
64681994630,12cyclictest12-21ksoftirqd/021:40:000
64681994629,11cyclictest12-21ksoftirqd/020:15:010
6469199455,6cyclictest131rcu_preempt20:00:003
64691994526,13cyclictest272-21dbus-daemon19:15:003
64691994515,22cyclictest131rcu_preempt00:35:023
64689994535,6cyclictest30-21ksoftirqd/200:00:002
64689994531,11cyclictest235-21systemd-journal22:43:402
64689994528,13cyclictest272-21dbus-daemon00:10:002
64685994529,12cyclictest412-21sshd20:49:411
6468199456,9cyclictest12-21ksoftirqd/000:20:000
6468199455,9cyclictest12-21ksoftirqd/020:05:010
64681994537,4cyclictest69026-21latency_hist20:35:020
64681994536,6cyclictest12-21ksoftirqd/022:50:020
64681994534,8cyclictest12-21ksoftirqd/019:25:010
6468199452,7cyclictest12-21ksoftirqd/000:40:000
6468199451,8cyclictest0-21swapper/021:02:420
64681994514,10cyclictest12-21ksoftirqd/023:30:000
64681994510,6cyclictest131rcu_preempt22:06:410
64681994510,5cyclictest131rcu_preempt20:45:000
6469199449,6cyclictest131rcu_preempt23:30:013
6469199449,6cyclictest131rcu_preempt20:49:423
6469199441,8cyclictest0-21swapper/320:33:403
6469199441,8cyclictest0-21swapper/320:33:403
64689994430,11cyclictest1-21systemd21:50:002
64689994427,13cyclictest1-21systemd21:25:012
64689994426,14cyclictest272-21dbus-daemon00:05:022
64689994426,12cyclictest30-21ksoftirqd/221:40:002
64685994436,4cyclictest71953-21munin-run21:30:001
64685994427,13cyclictest363-21snmpd00:07:361
6468199446,5cyclictest131rcu_preempt21:58:420
6468199446,5cyclictest131rcu_preempt21:58:410
6468199445,6cyclictest12-21ksoftirqd/023:22:490
6468199445,6cyclictest12-21ksoftirqd/023:22:480
6468199444,9cyclictest12-21ksoftirqd/022:35:000
6468199441,7cyclictest12-21ksoftirqd/021:15:000
6469199438,5cyclictest131rcu_preempt00:00:013
6469199434,5cyclictest131rcu_preempt22:04:593
64691994317,5cyclictest131rcu_preempt20:20:013
64691994316,21cyclictest131rcu_preempt19:40:013
64689994329,11cyclictest30-21ksoftirqd/219:45:002
6468199439,5cyclictest131rcu_preempt19:55:000
6468199436,27cyclictest19050irq/34-mmc119:45:030
6468199434,7cyclictest12-21ksoftirqd/020:40:010
64681994332,8cyclictest12-21ksoftirqd/021:35:020
64681994332,8cyclictest12-21ksoftirqd/021:35:020
6468199431,5cyclictest0-21swapper/020:31:400
64681994310,5cyclictest131rcu_preempt00:26:400
64691994232,7cyclictest69248-21sshd20:36:413
64691994232,6cyclictest78699-21munin-plugin-st23:40:013
64691994232,6cyclictest78699-21munin-plugin-st23:40:003
64691994231,8cyclictest75041-21sshd22:27:313
64691994225,13cyclictest37-21ksoftirqd/322:35:003
64691994224,13cyclictest235-21systemd-journal21:21:423
64691994221,5cyclictest37-21ksoftirqd/323:25:013
64691994221,5cyclictest37-21ksoftirqd/323:25:003
64689994224,14cyclictest67211-21systemctl20:00:002
64689994210,24cyclictest131rcu_preempt22:27:392
64685994230,9cyclictest363-21snmpd20:22:151
6468199427,6cyclictest131rcu_preempt00:25:010
6468199427,5cyclictest131rcu_preempt20:57:150
6468199424,7cyclictest12-21ksoftirqd/022:30:000
64681994230,7cyclictest151rcuc/023:00:010
64681994223,14cyclictest66129-21sshd19:35:400
64681994214,5cyclictest131rcu_preempt20:55:020
64681994212,6cyclictest131rcu_preempt23:35:020
64681994212,5cyclictest131rcu_preempt23:44:590
6469199414,7cyclictest37-21ksoftirqd/320:04:403
64691994129,6cyclictest70300-21cron20:59:593
6469199412,5cyclictest131rcu_preempt19:20:003
64691994122,13cyclictest272-21dbus-daemon21:50:003
6469199411,4cyclictest0-21swapper/322:09:403
64691994113,6cyclictest131rcu_preempt19:55:013
6469199411,10cyclictest131rcu_preempt20:55:013
64689994129,9cyclictest79487-21systemd-run23:55:022
64689994129,9cyclictest79487-21systemd-run23:55:012
64689994128,9cyclictest306-21in:imuxsock22:22:412
64689994127,9cyclictest235-21systemd-journal19:40:002
64689994127,10cyclictest235-21systemd-journal21:56:402
64689994127,10cyclictest235-21systemd-journal21:56:402
64689994126,10cyclictest272-21dbus-daemon20:35:022
64689994126,10cyclictest272-21dbus-daemon20:35:012
64689994116,9cyclictest131rcu_preempt19:10:012
64689994111,10cyclictest30-21ksoftirqd/220:45:012
6468199417,5cyclictest131rcu_preempt21:55:010
64681994134,5cyclictest12-21ksoftirqd/022:20:020
64681994123,11cyclictest272-21dbus-daemon19:30:000
64681994112,8cyclictest12-21ksoftirqd/022:53:180
6469199407,5cyclictest131rcu_preempt21:44:413
6469199407,5cyclictest131rcu_preempt21:44:403
6469199402,8cyclictest37-21ksoftirqd/320:45:023
6469199402,8cyclictest37-21ksoftirqd/319:43:393
64691994027,7cyclictest71157-21munin-plugin-st21:14:593
6469199401,9cyclictest0-21swapper/322:16:413
64691994016,20cyclictest37-21ksoftirqd/321:27:203
64691994016,20cyclictest37-21ksoftirqd/319:22:403
64691994013,5cyclictest131rcu_preempt23:00:013
6469199401,19cyclictest0-21swapper/322:25:023
64691994010,4cyclictest131rcu_preempt21:52:243
6468999404,5cyclictest131rcu_preempt20:40:012
64689994030,7cyclictest80735-21sshd00:16:412
6468999402,7cyclictest30-21ksoftirqd/221:45:012
6468999402,7cyclictest30-21ksoftirqd/221:45:002
64689994026,9cyclictest30-21ksoftirqd/223:20:012
6468999402,32cyclictest71296-21cat21:15:012
64689994023,11cyclictest272-21dbus-daemon19:55:022
64689994022,13cyclictest272-21dbus-daemon19:15:012
6468999401,5cyclictest131rcu_preempt22:20:012
6468999401,4cyclictest131rcu_preempt22:05:012
6468199402,7cyclictest12-21ksoftirqd/019:40:030
6468199402,5cyclictest131rcu_preempt20:30:020
6469199394,6cyclictest131rcu_preempt00:30:003
6469199394,5cyclictest131rcu_preempt23:35:013
64691993926,10cyclictest81216-21sed00:25:023
64691993925,10cyclictest71577-21sed21:20:023
6469199391,9cyclictest75331-21kworker/3:100:15:013
64691993919,5cyclictest37-21ksoftirqd/322:55:003
6469199391,8cyclictest0-21swapper/321:08:413
6469199391,8cyclictest0-21swapper/321:08:413
6469199391,7cyclictest0-21swapper/320:06:523
6469199391,5cyclictest131rcu_preempt23:50:003
6468999393,6cyclictest131rcu_preempt21:30:012
64689993926,10cyclictest30-21ksoftirqd/220:10:012
6468999392,5cyclictest131rcu_preempt20:50:002
64689993914,15cyclictest30-21ksoftirqd/220:55:032
64689993913,16cyclictest30-21ksoftirqd/223:10:002
64689993911,5cyclictest131rcu_preempt22:55:002
6468199398,10cyclictest12-21ksoftirqd/023:01:480
6468199393,11cyclictest12-21ksoftirqd/023:12:400
64681993927,9cyclictest151rcuc/000:05:010
6468199391,9cyclictest12-21ksoftirqd/023:52:490
6468199391,9cyclictest12-21ksoftirqd/023:52:480
6468199391,34cyclictest0-21swapper/022:15:010
6469199389,8cyclictest131rcu_preempt00:37:403
6469199388,19cyclictest131rcu_preempt20:30:033
6469199384,5cyclictest131rcu_preempt22:48:413
6469199382,4cyclictest131rcu_preempt23:13:413
64691993823,11cyclictest37-21ksoftirqd/323:08:413
64691993822,12cyclictest37-21ksoftirqd/323:40:023
64691993820,12cyclictest235-21systemd-journal23:55:023
64691993820,12cyclictest235-21systemd-journal23:55:013
64691993817,5cyclictest37-21ksoftirqd/323:02:453
64691993816,8cyclictest37-21ksoftirqd/322:00:023
64691993816,8cyclictest37-21ksoftirqd/322:00:013
6469199381,11cyclictest37-21ksoftirqd/322:45:013
64689993830,5cyclictest81060-21sh00:25:012
6468999382,6cyclictest131rcu_preempt00:14:402
64689993826,8cyclictest235-21systemd-journal00:25:022
64689993826,6cyclictest30-21ksoftirqd/223:15:002
64689993826,10cyclictest30-21ksoftirqd/220:12:402
64689993824,11cyclictest70531-21cut21:00:022
64689993823,11cyclictest235-21systemd-journal23:24:422
64689993823,11cyclictest235-21systemd-journal23:24:412
64689993822,11cyclictest1-21systemd19:35:012
64689993821,13cyclictest235-21systemd-journal21:04:412
64689993811,8cyclictest30-21ksoftirqd/222:50:012
64685993814,19cyclictest64745-21latency_hist19:10:011
6468199387,10cyclictest12-21ksoftirqd/000:11:270
64681993826,9cyclictest12-21ksoftirqd/021:25:020
64691993720,3cyclictest37-21ksoftirqd/321:35:033
64691993720,12cyclictest1-21systemd19:50:023
64689993725,9cyclictest30-21ksoftirqd/200:40:012
64689993723,11cyclictest272-21dbus-daemon23:40:022
64689993723,11cyclictest272-21dbus-daemon23:40:012
64689993723,10cyclictest235-21systemd-journal23:32:412
64689993721,12cyclictest73203-21sshd21:50:422
6468999371,5cyclictest131rcu_preempt20:25:012
64681993720,12cyclictest272-21dbus-daemon20:10:010
64691993616,17cyclictest37-21ksoftirqd/322:11:413
6468999367,6cyclictest131rcu_preempt23:30:012
6468999363,6cyclictest131rcu_preempt21:17:412
64689993626,7cyclictest65789-21latency_hist19:30:022
64689993623,9cyclictest272-21dbus-daemon23:05:012
6468999361,5cyclictest131rcu_preempt20:30:012
6468999361,5cyclictest131rcu_preempt20:03:402
64691993526,7cyclictest37-21ksoftirqd/319:10:013
6468999351,6cyclictest0-21swapper/219:45:402
6468999351,5cyclictest131rcu_preempt00:31:412
6468999341,6cyclictest0-21swapper/222:09:402
64691993212,9cyclictest37-21ksoftirqd/300:06:413
6468999325,5cyclictest131rcu_preempt19:21:562
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional