You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 00:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Dec 06, 2025 12:45:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1361249999153,31cyclictest37-21ksoftirqd/312:20:013
1361246997726,14cyclictest272-21dbus-daemon10:45:012
1361249997426,9cyclictest1375789-21cron11:49:593
136124999731,3cyclictest131rcu_preempt12:15:033
1361249996519,8cyclictest1373198-21systemctl11:00:013
1361235996246,10cyclictest1361240-21grep07:10:020
1361249996121,6cyclictest1367469-21sh09:10:003
1361242996149,9cyclictest0-21swapper/107:10:011
1361235996121,8cyclictest12-21ksoftirqd/010:25:000
1361249996020,9cyclictest37-21ksoftirqd/308:20:013
1361235995942,11cyclictest235-21systemd-journal12:40:010
1361249995820,10cyclictest1362910-21sed07:40:023
1361249995736,15cyclictest235-21systemd-journal08:55:013
1361249995716,12cyclictest1366458-21cat08:50:023
1361249995620,10cyclictest1373958-21cron11:15:013
136124999552,17cyclictest1368276-21munin-run09:25:003
1361249995422,6cyclictest1361697-21latency_hist07:20:023
1361249995422,4cyclictest1375546-21latency_hist11:45:033
1361249995422,4cyclictest1375546-21latency_hist11:45:023
1361242995432,16cyclictest235-21systemd-journal09:45:001
1361249995324,11cyclictest37-21ksoftirqd/308:40:023
136123599532,16cyclictest281-21systemd-logind09:05:000
1361249995224,8cyclictest37-21ksoftirqd/311:10:003
1361246995129,16cyclictest1-21systemd11:05:012
136123599515,29cyclictest1377349-21modprobe12:20:000
1361235995147,2cyclictest0-21swapper/011:05:010
1361249995021,6cyclictest37-21ksoftirqd/311:55:023
136124999502,10cyclictest0-21swapper/309:45:003
1361246995032,11cyclictest272-21dbus-daemon09:40:002
136124699502,42cyclictest272-21dbus-daemon10:30:002
136124299501,5cyclictest131rcu_preempt08:00:001
136123599504,11cyclictest12-21ksoftirqd/011:50:010
1361249994929,15cyclictest272-21dbus-daemon10:50:013
1361249994926,5cyclictest412-21sshd11:30:443
136124699491,33cyclictest1366646-21kworker/u8:109:10:012
136124999485,31cyclictest307-21in:imklog09:15:023
1361246994835,9cyclictest235-21systemd-journal08:30:002
1361246994831,13cyclictest235-21systemd-journal07:14:412
1361242994833,10cyclictest1-21systemd07:55:001
1361242994830,10cyclictest23-21ksoftirqd/108:05:011
136123599488,9cyclictest1-21systemd08:50:010
136123599487,5cyclictest131rcu_preempt09:45:010
1361235994835,9cyclictest16950irq/224-800000009:10:000
1361235994812,6cyclictest131rcu_preempt07:50:010
136124699474,3cyclictest131rcu_preempt09:00:022
136124299473,9cyclictest23-21ksoftirqd/112:25:031
1361242994732,11cyclictest1-21systemd10:25:001
1361242994729,14cyclictest235-21systemd-journal10:55:011
136123599475,7cyclictest0-21swapper/008:05:020
1361235994737,7cyclictest12-21ksoftirqd/007:45:010
136124999468,10cyclictest37-21ksoftirqd/307:55:003
1361249994631,11cyclictest235-21systemd-journal12:30:013
136124999461,4cyclictest131rcu_preempt07:48:573
1361242994626,14cyclictest235-21systemd-journal08:15:001
136124299462,27cyclictest221rcuc/108:40:021
1361235994637,5cyclictest16950irq/224-800000009:00:000
1361235994637,5cyclictest0-21swapper/008:10:000
1361235994627,12cyclictest1372141-21cron10:40:000
136124999454,3cyclictest131rcu_preempt12:00:013
136124999454,3cyclictest131rcu_preempt12:00:003
1361249994537,4cyclictest0-21swapper/307:10:023
1361249994530,11cyclictest1-21systemd09:20:013
136124699455,8cyclictest30-21ksoftirqd/210:10:002
1361246994530,11cyclictest30-21ksoftirqd/211:00:002
1361246994527,14cyclictest1-21systemd07:35:012
1361246994525,14cyclictest272-21dbus-daemon10:20:012
136124699451,4cyclictest131rcu_preempt09:05:002
1361242994536,7cyclictest23-21ksoftirqd/109:25:011
136124299451,4cyclictest131rcu_preempt11:20:011
1361242994510,25cyclictest235-21systemd-journal09:15:021
136123599459,5cyclictest131rcu_preempt08:40:000
1361235994530,11cyclictest235-21systemd-journal07:35:000
1361235994511,7cyclictest131rcu_preempt10:00:000
1361249994413,6cyclictest131rcu_preempt10:15:003
1361246994428,12cyclictest1362742-21systemctl07:40:002
1361242994433,8cyclictest1376551-21sshd12:00:431
136124299442,5cyclictest131rcu_preempt09:05:011
136123599444,9cyclictest12-21ksoftirqd/011:25:010
136123599443,8cyclictest12-21ksoftirqd/007:23:020
136123599443,37cyclictest12-21ksoftirqd/009:40:010
1361235994432,7cyclictest1375272-21sendmail11:40:000
1361235994431,10cyclictest1366165-21cron08:45:000
1361235994428,11cyclictest235-21systemd-journal11:35:010
1361235994415,5cyclictest131rcu_preempt10:17:000
1361235994414,5cyclictest131rcu_preempt11:45:010
1361235994414,5cyclictest131rcu_preempt11:45:010
1361235994414,5cyclictest131rcu_preempt07:55:000
1361235994411,29cyclictest12-21ksoftirqd/011:25:170
1361235994411,29cyclictest12-21ksoftirqd/011:25:170
1361246994328,12cyclictest235-21systemd-journal07:42:432
1361246994327,11cyclictest235-21systemd-journal08:00:002
1361246994325,13cyclictest1-21systemd09:35:012
136124699431,4cyclictest131rcu_preempt09:50:022
1361242994325,13cyclictest235-21systemd-journal10:45:011
1361242994319,19cyclictest23-21ksoftirqd/111:15:011
1361235994328,12cyclictest1364818-21sshd08:15:430
1361235994326,12cyclictest1361705-21systemctl07:20:000
136123599431,4cyclictest131rcu_preempt09:35:010
136124699421,9cyclictest363-21snmpd12:04:002
136124299423,5cyclictest1376311-21latency_hist12:00:021
136124299423,5cyclictest1376311-21latency_hist12:00:021
1361242994229,9cyclictest1-21systemd09:40:001
1361242994229,10cyclictest23-21ksoftirqd/111:10:011
1361242994227,10cyclictest1-21systemd12:15:011
1361242994226,11cyclictest235-21systemd-journal08:40:001
136123599429,3cyclictest131rcu_preempt07:15:010
136123599423,7cyclictest1373199-21munin-run11:00:010
1361235994234,6cyclictest12-21ksoftirqd/012:05:000
1361235994231,8cyclictest12-21ksoftirqd/010:55:010
136123599422,37cyclictest0-21swapper/007:40:010
1361235994222,15cyclictest235-21systemd-journal09:55:010
136123599421,8cyclictest1377886-21munin-run12:30:010
136124999413,5cyclictest131rcu_preempt07:25:013
1361249994131,7cyclictest37-21ksoftirqd/309:35:023
1361249994126,11cyclictest37-21ksoftirqd/308:15:013
136124999412,5cyclictest131rcu_preempt12:35:003
136124999412,3cyclictest131rcu_preempt09:30:033
136124999412,3cyclictest131rcu_preempt09:30:023
1361249994122,12cyclictest1371127-21systemctl10:20:013
1361249994119,16cyclictest272-21dbus-daemon09:55:013
136124699415,26cyclictest382-21ntpd10:40:032
1361246994127,10cyclictest30-21ksoftirqd/210:50:002
1361246994126,11cyclictest272-21dbus-daemon10:15:002
1361246994126,11cyclictest235-21systemd-journal07:48:432
1361246994126,11cyclictest1-21systemd08:55:002
1361242994128,9cyclictest272-21dbus-daemon11:25:001
1361235994132,6cyclictest12-21ksoftirqd/008:00:020
1361235994130,7cyclictest151rcuc/010:45:000
1361235994127,10cyclictest235-21systemd-journal11:08:430
1361235994126,10cyclictest1-21systemd07:30:010
136123599412,14cyclictest1376883-21latency_hist12:10:010
1361235994120,17cyclictest1371657-21true10:30:000
136123599411,6cyclictest1377090-21munin-run12:15:010
1361235994111,8cyclictest12-21ksoftirqd/010:50:010
136124999403,4cyclictest131rcu_preempt08:00:003
1361249994025,10cyclictest37-21ksoftirqd/310:00:003
136124699406,6cyclictest131rcu_preempt10:33:432
1361246994032,4cyclictest1368525-21sh09:30:002
1361246994032,4cyclictest1368525-21sh09:30:002
1361242994028,9cyclictest23-21ksoftirqd/112:35:001
1361242994026,9cyclictest306-21in:imuxsock07:45:001
1361242994018,18cyclictest23-21ksoftirqd/111:05:011
1361242994017,19cyclictest23-21ksoftirqd/110:10:011
1361242994016,20cyclictest23-21ksoftirqd/111:45:011
1361242994016,20cyclictest23-21ksoftirqd/111:45:011
136123599407,6cyclictest131rcu_preempt11:52:440
1361235994031,7cyclictest12-21ksoftirqd/008:35:020
1361235994031,6cyclictest1365154-21unin-run08:25:000
1361235994030,6cyclictest12-21ksoftirqd/009:50:010
1361235994024,11cyclictest272-21dbus-daemon09:25:010
1361235994010,5cyclictest131rcu_preempt10:15:000
136124999395,4cyclictest131rcu_preempt10:10:013
136124999391,4cyclictest131rcu_preempt09:50:013
136124999391,3cyclictest131rcu_preempt10:40:003
1361249993913,6cyclictest131rcu_preempt11:40:003
1361246993930,5cyclictest1366430-21munin-plugin-st08:50:002
1361246993928,8cyclictest30-21ksoftirqd/211:25:002
1361246993927,8cyclictest1375537-21cron11:45:012
1361246993927,8cyclictest1375537-21cron11:45:002
1361242993929,7cyclictest1367443-21sshd09:07:431
136124299392,5cyclictest131rcu_preempt08:10:001
1361242993925,9cyclictest23-21ksoftirqd/111:34:591
1361242993921,14cyclictest1-21systemd10:40:001
1361242993921,13cyclictest1-21systemd08:20:011
136124299391,5cyclictest131rcu_preempt10:15:021
136124299391,5cyclictest131rcu_preempt09:50:031
1361235993930,6cyclictest19050irq/34-mmc109:30:020
1361235993930,6cyclictest19050irq/34-mmc109:30:010
1361235993927,9cyclictest1365377-21sshd08:28:440
1361235993926,9cyclictest1367972-21sshd09:17:430
1361235993924,10cyclictest1-21systemd11:15:010
1361235993913,6cyclictest12-21ksoftirqd/010:35:020
1361235993912,5cyclictest131rcu_preempt08:55:030
1361235993911,6cyclictest131rcu_preempt10:05:000
136124999383,3cyclictest131rcu_preempt11:20:013
1361249993825,8cyclictest1361418-21cron07:15:003
136124999382,4cyclictest131rcu_preempt10:30:023
1361249993819,14cyclictest235-21systemd-journal11:30:003
1361249993819,14cyclictest235-21systemd-journal11:30:003
1361246993831,4cyclictest1364863-21sendmail08:20:002
1361246993827,8cyclictest306-21in:imuxsock09:44:432
1361246993826,9cyclictest1364587-21cron08:15:002
1361246993825,9cyclictest30-21ksoftirqd/212:40:002
136124699381,33cyclictest48-21kcompactd011:47:252
136124299388,11cyclictest23-21ksoftirqd/111:30:021
136124299388,11cyclictest23-21ksoftirqd/111:30:011
1361242993826,8cyclictest1365610-21sshd08:30:431
1361242993826,8cyclictest0-21swapper/107:20:031
1361242993825,10cyclictest1371880-21sshd10:33:431
136123599385,6cyclictest131rcu_preempt11:19:590
1361235993824,10cyclictest1364584-21sshd08:14:430
1361235993810,5cyclictest131rcu_preempt12:35:000
136124999373,7cyclictest0-21swapper/307:30:023
1361249993725,9cyclictest37-21ksoftirqd/307:45:023
1361249993721,11cyclictest272-21dbus-daemon09:00:013
1361249993720,13cyclictest272-21dbus-daemon12:25:003
1361249993720,12cyclictest272-21dbus-daemon08:05:013
1361249993720,12cyclictest1-21systemd08:25:003
1361246993729,5cyclictest271-21cron12:30:002
1361246993721,12cyclictest1361932-21sed07:20:032
1361246993720,11cyclictest0-21swapper/211:40:012
136124299373,5cyclictest131rcu_preempt10:00:001
1361242993729,5cyclictest1-21systemd07:48:561
1361242993727,7cyclictest1376597-21kworker/1:112:30:011
1361242993726,8cyclictest23-21ksoftirqd/107:35:401
1361242993726,8cyclictest1367974-21sshd09:18:431
1361242993724,10cyclictest1375278-21systemctl11:40:001
1361242993723,9cyclictest1375795-21modprobe11:50:001
136124299371,7cyclictest131rcu_preempt11:55:001
136123599377,5cyclictest131rcu_preempt12:00:010
136123599377,5cyclictest131rcu_preempt12:00:010
136123599376,6cyclictest131rcu_preempt12:25:010
136123599375,6cyclictest131rcu_preempt10:10:000
136124999363,4cyclictest131rcu_preempt12:10:003
1361249993626,8cyclictest37-21ksoftirqd/309:05:013
1361249993624,9cyclictest37-21ksoftirqd/312:40:013
1361249993624,10cyclictest37-21ksoftirqd/310:25:003
1361249993621,11cyclictest1-21systemd10:35:023
1361249993620,12cyclictest235-21systemd-journal10:51:173
1361246993627,6cyclictest1365229-21latency_hist08:25:012
1361246993627,5cyclictest1365901-21sendmail08:40:002
1361246993626,6cyclictest1365745-21latency_hist08:35:022
1361246993625,8cyclictest1373902-21sort11:10:022
1361246993623,9cyclictest1369836-21wc09:55:012
1361246993623,8cyclictest272-21dbus-daemon10:25:012
1361246993622,10cyclictest1377360-21run-parts12:20:012
1361246993619,13cyclictest235-21systemd-journal08:05:012
1361246993619,11cyclictest235-21systemd-journal09:19:592
136124699361,32cyclictest30-21ksoftirqd/212:10:022
136124299365,3cyclictest131rcu_preempt12:20:001
136124299364,9cyclictest1371409-21kworker/1:210:50:011
1361235993627,6cyclictest12-21ksoftirqd/009:13:240
136124999352,4cyclictest131rcu_preempt08:40:003
136124999352,4cyclictest131rcu_preempt08:35:023
1361249993524,8cyclictest37-21ksoftirqd/310:05:013
136124999352,29cyclictest1376573-21cron12:04:593
136124999351,3cyclictest131rcu_preempt11:25:023
1361246993525,7cyclictest30-21ksoftirqd/208:10:002
1361246993523,9cyclictest30-21ksoftirqd/211:51:432
1361246993523,8cyclictest1363704-21grep07:55:022
1361246993520,10cyclictest30-21ksoftirqd/210:00:022
1361242993526,7cyclictest23-21ksoftirqd/108:25:021
1361242993526,7cyclictest23-21ksoftirqd/107:35:011
1361242993525,8cyclictest23-21ksoftirqd/108:55:011
1361242993525,5cyclictest23-21ksoftirqd/109:30:031
1361242993525,5cyclictest23-21ksoftirqd/109:30:021
1361242993522,8cyclictest23-21ksoftirqd/110:05:001
136124299352,17cyclictest131rcu_preempt09:35:011
136124299351,9cyclictest1371195-21latency_hist10:20:021
1361242993519,12cyclictest235-21systemd-journal09:00:011
1361242993519,12cyclictest1-21systemd11:00:011
1361242993512,5cyclictest131rcu_preempt07:25:001
1361242993511,17cyclictest1366423-21modprobe08:50:011
1361249993423,8cyclictest37-21ksoftirqd/311:05:013
136124999341,5cyclictest131rcu_preempt08:10:003
136124999341,3cyclictest131rcu_preempt08:30:013
136124699344,3cyclictest131rcu_preempt08:45:012
1361246993427,5cyclictest30-21ksoftirqd/212:00:032
1361246993427,5cyclictest30-21ksoftirqd/212:00:022
1361246993424,7cyclictest30-21ksoftirqd/212:35:002
1361246993424,7cyclictest1370322-21sshd10:01:432
1361246993424,6cyclictest306-21in:imuxsock07:30:012
1361246993420,11cyclictest1-21systemd11:30:022
1361246993420,11cyclictest1-21systemd11:30:012
1361246993418,13cyclictest30-21ksoftirqd/211:19:592
1361242993411,19cyclictest131rcu_preempt07:15:011
1361249993326,4cyclictest1372430-21latency_hist10:45:023
136124699333,3cyclictest131rcu_preempt09:25:012
1361246993325,5cyclictest1375014-21latency_hist11:35:012
1361246993318,11cyclictest235-21systemd-journal09:15:002
136124299332,6cyclictest0-21swapper/110:30:031
136124299332,5cyclictest131rcu_preempt08:30:011
1361242993322,8cyclictest1362358-21sed07:30:021
1361249993224,5cyclictest1368791-21kworker/3:109:40:023
136124699322,5cyclictest131rcu_preempt11:15:012
1361246993222,7cyclictest1372904-21sshd10:51:172
1361242993224,3cyclictest23-21ksoftirqd/109:55:001
1361249993115,11cyclictest1-21systemd07:35:013
1361246993121,7cyclictest30-21ksoftirqd/207:25:012
1361246993119,9cyclictest1361389-21sed07:10:022
1361246993021,7cyclictest30-21ksoftirqd/212:25:012
1361242993010,7cyclictest23-21ksoftirqd/112:38:431
136124699293,3cyclictest131rcu_preempt12:15:012
1361242992821,5cyclictest23-21ksoftirqd/112:10:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional