You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-15 - 22:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Tue Jul 15, 2025 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27827199742,67cyclictest48-21kcompactd011:55:131
27826599741,48cyclictest281294-21modprobe08:06:350
27826599732,36cyclictest235-21systemd-journal10:20:000
27826599711,65cyclictest279702-21latency_hist07:35:020
27826599701,30cyclictest279015-21unin-run07:25:010
27826599692,63cyclictest1-21systemd11:26:350
27827199682,61cyclictest294353-21cron12:20:011
27826599682,62cyclictest278432-21sshd07:12:360
27826599681,63cyclictest288381-21sshd10:21:340
27826599671,36cyclictest0-21swapper/012:07:360
27826599662,59cyclictest281862-21sshd08:18:360
27826599662,59cyclictest281862-21sshd08:18:350
27826599661,61cyclictest150-21pr/ttyS210:06:220
27827199652,12cyclictest288902-21cron10:35:001
278271996518,41cyclictest235-21systemd-journal12:10:021
27826599657,53cyclictest278267-21cat07:10:020
278265996529,31cyclictest295139-21cron12:35:000
27826599652,59cyclictest294120-21munin-run12:15:010
27826599652,58cyclictest288916-21munin-run10:35:000
278265996524,38cyclictest282599-21latency_hist08:30:020
278265996524,38cyclictest282599-21latency_hist08:30:020
27826599651,60cyclictest291495-21sshd11:20:350
27826599642,57cyclictest285560-21munin-run09:30:020
27826599642,57cyclictest285560-21munin-run09:30:010
278265996420,40cyclictest292657-21sed11:45:020
27826599641,58cyclictest292270-21sshd11:39:350
27826599641,58cyclictest290461-21sshd11:03:360
27827199633,17cyclictest281-21systemd-logind08:40:011
278271996328,31cyclictest283457-21/usr/sbin/munin08:50:011
27827199632,15cyclictest290722-21cron11:10:011
27826599632,55cyclictest293572-21sshd12:01:350
27826599631,57cyclictest284313-21latency_hist09:05:020
27826599622,55cyclictest288664-21munin-run10:30:000
278265996224,34cyclictest292033-21latency_hist11:35:010
27826599621,57cyclictest1-21systemd10:36:360
27826599621,56cyclictest48-21kcompactd008:50:250
27826599621,55cyclictest48-21kcompactd007:43:380
27827199613,12cyclictest279722-21modprobe07:35:371
278271996125,31cyclictest289581-21latency_hist10:45:011
27827199612,16cyclictest295431-21sendmail-msp12:39:591
27827199612,11cyclictest285858-21cat09:35:011
278271996119,37cyclictest288409-21latency_hist10:25:021
278271996117,39cyclictest272-21dbus-daemon09:50:011
278271996115,41cyclictest281078-21munin-plugin-st08:05:001
278271996110,4cyclictest0-21swapper/112:10:001
27826599613,31cyclictest1-21systemd07:55:350
27826599612,54cyclictest294877-21sshd12:28:350
27826599611,56cyclictest282889-21sshd08:35:360
27826599611,55cyclictest289422-21sshd10:42:350
27826599611,55cyclictest279469-21sshd07:30:350
27826599611,55cyclictest278683-21cron07:17:000
27826599611,54cyclictest48-21kcompactd008:13:060
27827199605,10cyclictest285297-21grep09:25:011
278271996027,27cyclictest306-21in:imuxsock07:30:001
27827199602,18cyclictest292049-21munin-run11:35:021
278271996020,34cyclictest306-21in:imuxsock10:15:011
278271996019,36cyclictest292338-21latency_hist11:40:021
278271996015,39cyclictest0-21swapper/110:00:011
278271996015,39cyclictest0-21swapper/110:00:001
278271996010,9cyclictest278521-21grep07:15:011
27826599602,54cyclictest293305-21sshd11:59:350
27826599602,52cyclictest293034-21sshd11:50:330
27826599601,55cyclictest290718-21sshd11:09:350
27826599601,55cyclictest281059-21sshd08:04:420
27826599601,55cyclictest279224-21sshd07:27:360
27826599601,54cyclictest284976-21cron09:16:590
27826599601,53cyclictest48-21kcompactd010:57:300
27827599591,34cyclictest277371-21kworker/u8:307:35:032
27827599591,34cyclictest277371-21kworker/u8:307:35:022
27827199594,7cyclictest291242-21modprobe11:20:001
27827199593,14cyclictest290164-21modprobe10:55:031
27826599592,53cyclictest294659-21munin-run12:25:010
27826599592,53cyclictest287873-21munin-run10:15:010
27826599592,52cyclictest363-21snmpd08:47:040
27826599592,52cyclictest287329-21sshd10:00:040
27826599591,55cyclictest282636-21sshd08:30:360
27827199586,11cyclictest288663-21modprobe10:29:591
278271995839,14cyclictest1-21systemd11:05:001
27827199583,11cyclictest235-21systemd-journal11:45:001
27827199582,12cyclictest283697-21cron08:55:001
278271995812,7cyclictest279525-21cat07:35:011
278271995812,7cyclictest279525-21cat07:35:011
27827199581,16cyclictest287371-21true10:05:001
27826599581,53cyclictest363-21snmpd10:45:020
27826599581,53cyclictest286019-21sshd09:36:340
27826599581,52cyclictest48-21kcompactd009:52:030
27826599581,52cyclictest363-21snmpd09:05:310
27826599581,52cyclictest295602-21date12:40:020
27826599581,51cyclictest48-21kcompactd008:42:370
27826599581,51cyclictest48-21kcompactd008:42:360
27827199578,7cyclictest0-21swapper/111:50:021
27827199574,6cyclictest284483-21modprobe09:10:011
27827199573,8cyclictest1-21systemd09:05:001
278271995728,25cyclictest295072-21latency_hist12:30:021
27827199572,18cyclictest281977-21wc08:20:021
27827199572,18cyclictest281977-21wc08:20:011
27827199572,12cyclictest281716-21latency_hist08:15:021
278271995718,34cyclictest282598-21wc08:30:031
278271995718,34cyclictest282598-21wc08:30:021
278271995715,37cyclictest306-21in:imuxsock11:15:011
27826599571,52cyclictest0-21swapper/009:56:340
27826599571,52cyclictest0-21swapper/009:56:340
27826599571,50cyclictest48-21kcompactd008:56:330
27826599571,50cyclictest48-21kcompactd008:56:320
27827199563,16cyclictest295158-21true12:35:011
27827199562,13cyclictest293071-21munin-run11:54:591
27827199561,13cyclictest291786-21sh11:30:001
278265995631,22cyclictest1-21systemd09:15:010
27826599562,49cyclictest291223-21sshd11:17:350
278265995615,37cyclictest279927-21latency_hist07:40:010
27826599561,50cyclictest48-21kcompactd007:52:310
27826599561,50cyclictest363-21snmpd09:34:550
27827199553,8cyclictest1-21systemd10:40:021
27826599551,49cyclictest48-21kcompactd009:48:090
27826599551,49cyclictest363-21snmpd09:22:230
27826599551,49cyclictest363-21snmpd07:48:220
27826599551,49cyclictest363-21snmpd07:48:220
27827199549,8cyclictest308-21rs:main1
27827199542,9cyclictest235-21systemd-journal10:10:011
27827199542,14cyclictest282673-21munin-run08:34:591
27827199542,13cyclictest293615-21munin-run12:05:021
278271995412,4cyclictest0-21swapper/111:20:191
27826599541,49cyclictest363-21snmpd12:19:440
27826599541,49cyclictest363-21snmpd09:40:550
27826599541,49cyclictest363-21snmpd09:40:540
27827599531,6cyclictest131rcu_preempt08:35:022
27827199534,10cyclictest280555-21unin-run07:55:001
27827199533,9cyclictest286035-21cron09:40:001
27827199531,14cyclictest278732-21munin-run07:20:011
27827199531,12cyclictest289711-21munin-run10:50:011
27826599532,47cyclictest290961-21sshd11:12:360
27826599531,48cyclictest48-21kcompactd008:20:220
27826599531,48cyclictest363-21snmpd11:45:040
27827199522,9cyclictest283973-21grep09:00:011
27827199522,9cyclictest283973-21grep09:00:001
27827199522,15cyclictest279014-21systemd-run07:25:001
27827199522,14cyclictest280265-21sshd07:48:371
27827199522,14cyclictest280265-21sshd07:48:361
27827199522,13cyclictest290087-21latency_hist10:55:021
27827199522,13cyclictest290087-21latency_hist10:55:021
27827199522,11cyclictest1-21systemd07:45:001
27828099517,6cyclictest131rcu_preempt10:40:013
27827199514,9cyclictest0-21swapper/109:30:021
27827199514,9cyclictest0-21swapper/109:30:011
27827199514,8cyclictest282167-21munin-run08:24:591
27827199514,8cyclictest280780-21cron08:00:001
27827199512,14cyclictest284743-21cron09:15:001
27826599511,46cyclictest289925-21sshd10:52:350
27826599511,46cyclictest289925-21sshd10:52:340
27828099495,36cyclictest37-21ksoftirqd/311:00:023
278275994933,12cyclictest272-21dbus-daemon10:50:012
278275994915,6cyclictest131rcu_preempt12:25:002
27827199493,10cyclictest283193-21modprobe08:45:001
27827199493,10cyclictest283193-21modprobe08:45:001
27827199491,13cyclictest294634-21cron12:25:001
27828099486,7cyclictest37-21ksoftirqd/311:10:013
278280994833,11cyclictest1-21systemd12:30:013
27828099481,4cyclictest131rcu_preempt10:00:003
27828099481,4cyclictest131rcu_preempt10:00:003
278280994814,7cyclictest37-21ksoftirqd/311:24:593
27827199484,9cyclictest281318-21modprobe08:09:591
27827199482,19cyclictest285025-21grep09:19:591
27827199481,14cyclictest286327-21modprobe09:45:011
27827199481,14cyclictest286327-21modprobe09:45:001
278280994712,5cyclictest131rcu_preempt09:45:023
278280994712,5cyclictest131rcu_preempt09:45:013
278275994730,13cyclictest235-21systemd-journal08:25:002
278280994610,5cyclictest131rcu_preempt10:05:023
278280994610,5cyclictest131rcu_preempt07:45:023
27827599469,5cyclictest131rcu_preempt08:45:032
27827599469,5cyclictest131rcu_preempt08:45:022
278275994631,10cyclictest235-21systemd-journal09:40:012
278275994617,7cyclictest30-21ksoftirqd/208:15:002
27827199462,14cyclictest286936-21wc09:55:011
27827199461,13cyclictest278331-21latency_hist07:10:011
27828099453,6cyclictest131rcu_preempt12:20:013
278280994510,5cyclictest131rcu_preempt10:50:003
278275994532,10cyclictest1-21systemd09:20:012
27828099449,5cyclictest131rcu_preempt09:35:013
278280994433,5cyclictest306-21in:imuxsock10:30:013
278280994431,9cyclictest1-21systemd07:35:023
278280994431,9cyclictest1-21systemd07:35:013
278280994430,11cyclictest1-21systemd08:00:003
278275994428,12cyclictest235-21systemd-journal10:15:002
27828099439,6cyclictest131rcu_preempt11:40:013
27828099438,5cyclictest131rcu_preempt10:17:353
27828099435,33cyclictest289460-21unin-run10:45:013
278280994329,10cyclictest1-21systemd08:30:013
278280994329,10cyclictest1-21systemd08:30:013
278280994327,11cyclictest272-21dbus-daemon07:20:013
27828099432,6cyclictest131rcu_preempt12:40:003
278280994310,6cyclictest131rcu_preempt07:55:013
27827599435,34cyclictest278811-21cat07:20:022
278275994329,10cyclictest235-21systemd-journal11:40:352
278275994328,12cyclictest272-21dbus-daemon07:40:002
27827599432,7cyclictest0-21swapper/207:59:362
278275994325,13cyclictest235-21systemd-journal08:40:002
278275994310,7cyclictest30-21ksoftirqd/211:40:012
27828099424,5cyclictest131rcu_preempt11:26:353
278280994232,7cyclictest235-21systemd-journal09:22:353
27827599424,5cyclictest131rcu_preempt07:54:592
278275994231,7cyclictest30-21ksoftirqd/212:20:032
278275994230,9cyclictest281050-21sshd08:03:362
278275994230,10cyclictest30-21ksoftirqd/208:20:012
278275994230,10cyclictest30-21ksoftirqd/208:20:012
27827599422,6cyclictest131rcu_preempt12:40:002
278275994223,13cyclictest272-21dbus-daemon11:35:012
27828099415,5cyclictest131rcu_preempt08:45:033
27828099415,5cyclictest131rcu_preempt08:45:023
278280994132,5cyclictest284481-21cron09:10:003
27828099412,5cyclictest294645-21munin-plugin-st12:25:003
278280994123,14cyclictest235-21systemd-journal08:36:363
278280994119,18cyclictest291233-21cron11:20:003
27828099411,6cyclictest0-21swapper/312:05:353
27828099411,4cyclictest131rcu_preempt08:20:023
27828099411,4cyclictest131rcu_preempt08:20:013
278275994131,8cyclictest30-21ksoftirqd/209:15:022
278275994127,10cyclictest30-21ksoftirqd/209:04:262
278275994125,14cyclictest30-21ksoftirqd/208:10:022
278275994122,15cyclictest1-21systemd07:50:022
278275994122,15cyclictest1-21systemd07:50:012
27828099408,5cyclictest131rcu_preempt10:06:353
27828099403,6cyclictest131rcu_preempt09:55:023
278280994024,12cyclictest286035-21cron09:40:003
278280994017,10cyclictest37-21ksoftirqd/309:14:303
278280994013,5cyclictest131rcu_preempt09:49:593
278280994012,4cyclictest131rcu_preempt08:11:363
27827599406,5cyclictest131rcu_preempt12:30:022
27827599404,6cyclictest131rcu_preempt12:05:022
278275994029,7cyclictest286317-21cron09:45:012
278275994029,7cyclictest286317-21cron09:45:002
278275994026,10cyclictest30-21ksoftirqd/210:40:012
278275994023,12cyclictest235-21systemd-journal10:25:002
278280993928,7cyclictest306-21in:imuxsock07:39:593
278280993918,15cyclictest272-21dbus-daemon07:25:013
278280993916,19cyclictest361rcuc/311:50:013
278280993914,6cyclictest131rcu_preempt12:13:353
278275993928,8cyclictest30-21ksoftirqd/210:59:592
278275993928,8cyclictest283467-21systemctl08:50:002
278275993925,10cyclictest30-21ksoftirqd/211:10:032
278275993922,11cyclictest272-21dbus-daemon10:05:012
278280993828,7cyclictest37-21ksoftirqd/310:15:013
278280993823,11cyclictest235-21systemd-journal11:35:003
278280993816,6cyclictest131rcu_preempt12:05:013
278280993814,5cyclictest131rcu_preempt11:01:363
27827599383,6cyclictest131rcu_preempt12:00:012
278275993828,7cyclictest30-21ksoftirqd/212:35:012
278275993827,8cyclictest30-21ksoftirqd/210:00:022
278275993827,8cyclictest30-21ksoftirqd/210:00:012
278275993827,8cyclictest283683-21sshd08:52:352
278275993826,9cyclictest30-21ksoftirqd/207:30:012
278275993826,9cyclictest30-21ksoftirqd/207:25:002
27828099377,4cyclictest131rcu_preempt12:00:003
27828099376,5cyclictest131rcu_preempt10:50:213
27828099376,5cyclictest131rcu_preempt10:50:203
27828099373,5cyclictest131rcu_preempt09:30:023
278280993729,6cyclictest37-21ksoftirqd/308:05:033
278280993729,5cyclictest284305-21latency_hist09:05:013
278280993728,6cyclictest37-21ksoftirqd/309:20:003
27828099371,5cyclictest131rcu_preempt07:50:013
27828099371,5cyclictest131rcu_preempt07:50:013
278275993724,10cyclictest272-21dbus-daemon11:30:002
278275993713,6cyclictest131rcu_preempt11:46:352
278275993712,6cyclictest131rcu_preempt10:20:012
278275993710,5cyclictest131rcu_preempt11:10:182
27828099362,5cyclictest131rcu_preempt09:00:013
27828099362,5cyclictest131rcu_preempt09:00:003
278280993625,8cyclictest37-21ksoftirqd/310:35:013
27828099362,4cyclictest131rcu_preempt11:50:283
278280993624,9cyclictest37-21ksoftirqd/308:09:593
278280993613,6cyclictest131rcu_preempt08:35:003
278280993611,6cyclictest131rcu_preempt11:45:013
278275993626,8cyclictest30-21ksoftirqd/208:55:032
278275993626,8cyclictest30-21ksoftirqd/208:55:022
278275993626,6cyclictest306-21in:imuxsock11:20:012
278275993615,5cyclictest131rcu_preempt11:24:382
27828099355,6cyclictest37-21ksoftirqd/308:45:363
278280993519,11cyclictest37-21ksoftirqd/308:55:013
27828099351,6cyclictest131rcu_preempt07:26:363
27827599356,5cyclictest131rcu_preempt09:49:592
278275993526,5cyclictest306-21in:imuxsock11:05:012
27827599352,5cyclictest131rcu_preempt09:35:012
278275993523,9cyclictest294299-21sed12:15:022
278275993522,10cyclictest30-21ksoftirqd/209:25:002
27827599351,4cyclictest131rcu_preempt10:53:352
27827599351,4cyclictest131rcu_preempt10:53:342
278275993511,6cyclictest131rcu_preempt10:30:002
27828099342,5cyclictest131rcu_preempt09:30:013
278280993421,9cyclictest278444-21sshd07:14:323
27827599344,5cyclictest131rcu_preempt12:05:362
27827599343,5cyclictest131rcu_preempt10:35:022
27827599342,4cyclictest131rcu_preempt10:09:242
278275993424,7cyclictest30-21ksoftirqd/211:55:022
278275993422,9cyclictest30-21ksoftirqd/207:15:012
278275993421,9cyclictest30-21ksoftirqd/209:30:012
278275993421,9cyclictest30-21ksoftirqd/209:30:002
27828099333,4cyclictest131rcu_preempt12:34:543
278280993310,5cyclictest131rcu_preempt10:24:253
27827599331,5cyclictest131rcu_preempt07:41:362
27827599328,5cyclictest131rcu_preempt08:30:012
27827599328,5cyclictest131rcu_preempt08:30:002
278275993220,9cyclictest235-21systemd-journal09:10:002
278280993124,4cyclictest282163-21latency_hist08:25:033
278275993123,5cyclictest306-21in:imuxsock09:55:002
27827599311,8cyclictest0-21swapper/210:45:012
27828099301,4cyclictest131rcu_preempt11:10:553
278275992921,5cyclictest278351-21latency_hist07:10:012
278280992617,6cyclictest37-21ksoftirqd/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional