You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-12 - 10:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Thu Sep 12, 2024 00:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
413090699891,49cyclictest0-21swapper/021:55:010
413090699871,3cyclictest0-21swapper/023:05:010
413090699861,55cyclictest0-21swapper/019:53:130
413090699851,3cyclictest0-21swapper/022:10:010
4130906998173,2cyclictest0-21swapper/022:25:010
413090699801,47cyclictest4138384-21munin-run21:35:010
4130906997365,2cyclictest0-21swapper/019:25:020
4130906997333,34cyclictest4141242-21latency_hist22:30:010
4130920997228,38cyclictest1-21systemd23:45:003
413090699722,66cyclictest4147633-21cat00:30:020
4130920996923,4cyclictest4132871-21latency_hist19:45:023
413090699692,62cyclictest4133443-21munin-run20:00:000
413090699691,64cyclictest0-21swapper/000:40:010
4130916996830,13cyclictest272-21dbus-daemon21:50:002
413090699671,61cyclictest4145821-21sshd23:55:130
4130906996629,33cyclictest4146645-21munin-plugin-st00:15:000
413090699652,58cyclictest4137870-21munin-run21:25:000
413090699651,60cyclictest4137573-21sshd21:19:140
413090699651,60cyclictest4137573-21sshd21:19:130
4130920996428,32cyclictest306-21in:imuxsock21:40:013
413090699642,58cyclictest4147149-21sshd00:20:120
413090699642,57cyclictest4139131-21sshd21:48:130
413090699641,59cyclictest0-21swapper/021:58:130
4130920996320,14cyclictest272-21dbus-daemon20:10:013
4130920996319,5cyclictest4141032-21latency_hist22:25:013
413090699633,54cyclictest4144551-21sshd23:32:580
413090699632,57cyclictest4143835-21munin-run23:20:010
413090699632,57cyclictest4142476-21sshd22:52:130
413090699632,57cyclictest4138665-21munin-run21:40:010
413090699632,56cyclictest4145084-21sshd23:43:130
413090699632,56cyclictest4143514-21sshd23:13:130
413090699632,56cyclictest4134972-21sshd20:26:130
413090699631,57cyclictest4136503-21sshd20:55:130
413090699631,57cyclictest4135486-21sshd20:38:130
4130920996222,8cyclictest37-21ksoftirqd/300:00:023
413090699622,56cyclictest4145331-21sshd23:48:140
413090699622,56cyclictest4133943-21sshd20:08:130
413090699622,55cyclictest4142255-21munin-run22:50:000
413090699622,54cyclictest4146378-21sshd00:09:480
413090699621,57cyclictest4143266-21sshd23:09:130
413090699621,57cyclictest0-21swapper/019:50:020
413090699621,56cyclictest4142724-21sshd22:56:130
413090699621,56cyclictest4133690-21sshd20:02:350
4130920996152,4cyclictest37-21ksoftirqd/322:55:023
4130920996124,33cyclictest4144574-21munin-plugin-st23:35:013
413090699612,54cyclictest4131343-21sshd19:17:270
413090699611,56cyclictest4145574-21sshd23:51:130
413090699611,56cyclictest4141701-21sshd22:37:520
413090699611,56cyclictest4141444-21sshd22:31:130
413090699611,56cyclictest4135225-21sshd20:31:130
413090699611,56cyclictest0-21swapper/023:26:130
413090699611,55cyclictest4136006-21sshd20:46:130
413090699601,55cyclictest4132111-21sshd19:30:130
413090699601,54cyclictest4144791-21sshd23:35:130
413090699601,54cyclictest4138901-21munin-run21:45:000
413092099596,16cyclictest4140284-21wc22:10:013
413092099593,18cyclictest235-21systemd-journal19:35:003
4130906995930,25cyclictest306-21in:imuxsock20:55:000
413090699592,53cyclictest4140420-21sshd22:14:130
413090699592,53cyclictest4132647-21sshd19:41:130
413090699591,54cyclictest363-21snmpd19:37:020
4130920995824,31cyclictest37-21ksoftirqd/300:35:033
4130920995824,31cyclictest37-21ksoftirqd/300:35:023
4130920995821,10cyclictest37-21ksoftirqd/321:41:133
4130916995843,11cyclictest1-21systemd19:30:012
4130916995843,11cyclictest1-21systemd19:30:012
413092099571,52cyclictest4133099-21sed19:50:023
413090699572,50cyclictest363-21snmpd21:12:440
413090699571,52cyclictest363-21snmpd20:20:090
413090699571,52cyclictest363-21snmpd20:20:090
413090699571,51cyclictest4140651-21modprobe22:15:130
413090699571,51cyclictest4134446-21sshd20:17:130
413090699571,51cyclictest363-21snmpd00:15:040
413090699571,51cyclictest363-21snmpd00:15:040
4130906995713,39cyclictest4137060-21modprobe21:10:000
413090699562,50cyclictest4134214-21munin-run20:15:000
413090699562,50cyclictest363-21snmpd23:22:150
413090699561,50cyclictest363-21snmpd21:26:440
4130906995611,42cyclictest0-21swapper/000:30:150
4130906995611,42cyclictest0-21swapper/000:30:140
4130920995517,6cyclictest4131126-21latency_hist19:15:013
413090699553,47cyclictest363-21snmpd00:04:440
413090699552,48cyclictest363-21snmpd20:40:310
413090699552,48cyclictest363-21snmpd20:40:310
413090699551,49cyclictest4139896-21sshd22:00:130
4130920995422,6cyclictest4144321-21latency_hist23:30:023
413090699543,46cyclictest363-21snmpd22:41:530
413090699541,48cyclictest363-21snmpd19:27:350
413090699541,48cyclictest363-21snmpd19:27:350
413090699541,48cyclictest363-21snmpd19:13:280
4130920995318,11cyclictest1-21systemd23:25:013
413090699532,46cyclictest363-21snmpd21:00:050
4130920995221,10cyclictest37-21ksoftirqd/321:00:023
4130920995214,8cyclictest272-21dbus-daemon23:55:013
4130910995216,27cyclictest4145847-21modprobe00:00:011
413092099513,11cyclictest4145357-21cut23:50:013
413092099502,16cyclictest4146929-21sendmail-msp00:20:003
413092099502,16cyclictest4146929-21sendmail-msp00:20:003
4130920995019,4cyclictest4143796-21latency_hist23:20:013
4130920995017,8cyclictest4141604-21sed22:35:023
4130920995017,8cyclictest37-21ksoftirqd/323:10:023
4130920995012,8cyclictest235-21systemd-journal22:20:013
4130920994920,7cyclictest37-21ksoftirqd/320:45:023
4130920994920,7cyclictest37-21ksoftirqd/320:45:023
4130920994918,7cyclictest37-21ksoftirqd/322:05:013
4130920994912,12cyclictest272-21dbus-daemon21:35:013
413092099491,19cyclictest4134483-21sendmail20:20:003
4130920994910,12cyclictest1-21systemd19:30:013
4130920994910,12cyclictest1-21systemd19:30:013
4130910994933,12cyclictest235-21systemd-journal19:15:011
4130920994817,7cyclictest4137240-21cut21:10:023
413091699487,9cyclictest30-21ksoftirqd/219:30:132
4130916994834,10cyclictest272-21dbus-daemon00:00:002
4130910994832,10cyclictest23-21ksoftirqd/120:55:021
4130916994734,8cyclictest4142758-21sendmail23:00:002
4130916994732,12cyclictest272-21dbus-daemon20:50:002
4130916994729,14cyclictest272-21dbus-daemon19:20:012
4130910994734,6cyclictest306-21in:imuxsock20:10:011
4130910994725,17cyclictest235-21systemd-journal19:20:001
4130920994634,9cyclictest4146618-21sshd00:12:133
4130920994630,12cyclictest272-21dbus-daemon22:45:003
4130916994529,11cyclictest1-21systemd00:30:002
4130916994528,12cyclictest235-21systemd-journal20:40:002
4130910994529,13cyclictest235-21systemd-journal22:25:001
413090699453,35cyclictest4130907-21latency_hist19:10:020
4130916994429,11cyclictest1-21systemd22:45:012
4130920994329,11cyclictest4146311-21sed00:05:033
4130916994326,13cyclictest272-21dbus-daemon20:25:012
4130916994326,13cyclictest272-21dbus-daemon20:25:012
413091099437,22cyclictest235-21systemd-journal21:00:001
4130910994331,9cyclictest4130919-21sort19:10:021
4130910994330,10cyclictest1-21systemd22:10:001
4130910994328,9cyclictest23-21ksoftirqd/100:25:011
4130910994319,18cyclictest272-21dbus-daemon19:45:011
4130910994316,21cyclictest4139153-21grep21:50:001
413091099431,3cyclictest131rcu_preempt21:45:011
4130920994229,9cyclictest272-21dbus-daemon21:30:013
4130920994228,9cyclictest1-21systemd19:20:013
4130920994228,11cyclictest272-21dbus-daemon20:15:003
413091699426,4cyclictest131rcu_preempt21:15:002
413091699423,28cyclictest4136538-21modprobe21:00:002
4130916994229,10cyclictest30-21ksoftirqd/200:10:002
4130916994228,10cyclictest1-21systemd00:15:012
413091699422,4cyclictest131rcu_preempt23:15:002
413091699422,3cyclictest131rcu_preempt21:10:002
413091099428,19cyclictest4134469-21modprobe20:20:001
413091099427,5cyclictest131rcu_preempt00:40:001
4130910994228,9cyclictest235-21systemd-journal23:35:001
4130910994226,10cyclictest235-21systemd-journal22:30:031
4130920994127,9cyclictest0-21swapper/320:55:013
413091699413,3cyclictest131rcu_preempt23:40:012
4130916994128,8cyclictest30-21ksoftirqd/220:10:032
4130916994127,11cyclictest30-21ksoftirqd/222:35:022
4130916994127,10cyclictest30-21ksoftirqd/221:40:002
413091699412,3cyclictest131rcu_preempt00:05:002
4130910994127,11cyclictest23-21ksoftirqd/120:50:001
4130910994124,13cyclictest272-21dbus-daemon20:05:011
4130910994121,15cyclictest272-21dbus-daemon00:10:001
413092099404,7cyclictest131rcu_preempt20:00:003
4130920994029,7cyclictest4141691-21sshd22:35:143
4130916994028,8cyclictest306-21in:imuxsock22:15:012
4130916994027,10cyclictest4139631-21sshd21:57:472
413091699402,4cyclictest131rcu_preempt19:55:002
413091699402,3cyclictest131rcu_preempt20:55:002
4130916994016,17cyclictest272-21dbus-daemon23:30:012
413091699401,4cyclictest131rcu_preempt21:35:002
4130916994010,5cyclictest131rcu_preempt23:17:132
4130910994029,5cyclictest4138611-21cron21:40:001
4130910994023,12cyclictest235-21systemd-journal00:20:011
4130910994023,12cyclictest235-21systemd-journal00:20:011
4130910994022,12cyclictest1-21systemd22:00:011
4130910994021,14cyclictest1-21systemd22:55:011
4130910994020,12cyclictest306-21in:imuxsock21:30:011
4130910994017,16cyclictest272-21dbus-daemon20:40:011
4130920993928,8cyclictest4135228-21sshd20:32:133
4130916993927,9cyclictest4134194-21sshd20:13:132
4130916993927,7cyclictest4144060-21cron23:25:002
4130916993924,12cyclictest4134701-21sed20:20:022
413091699392,10cyclictest4141727-21latency_hist22:40:002
4130910993928,6cyclictest306-21in:imuxsock19:35:001
4130910993926,9cyclictest272-21dbus-daemon19:40:011
4130910993923,11cyclictest306-21in:imuxsock23:20:021
4130910993912,20cyclictest4140682-21cron22:20:001
413092099387,3cyclictest131rcu_preempt19:40:003
4130920993829,5cyclictest306-21in:imuxsock20:50:003
4130920993826,9cyclictest4147905-21sshd00:35:323
413091699385,3cyclictest131rcu_preempt19:25:002
413091699382,7cyclictest30-21ksoftirqd/221:27:132
4130916993826,9cyclictest4143256-21sshd23:07:132
413091699382,3cyclictest131rcu_preempt22:20:002
4130916993821,11cyclictest272-21dbus-daemon22:50:012
4130916993819,13cyclictest272-21dbus-daemon21:05:012
413091699381,3cyclictest131rcu_preempt19:50:012
413091099386,18cyclictest4147674-21modprobe00:35:001
413091099386,18cyclictest4147674-21modprobe00:35:001
413091099385,24cyclictest4140456-21cat22:15:011
413091099383,3cyclictest131rcu_preempt22:55:021
4130910993823,10cyclictest23-21ksoftirqd/100:15:001
4130910993822,9cyclictest4143781-21cron23:20:001
4130910993822,12cyclictest272-21dbus-daemon20:15:011
4130920993728,7cyclictest37-21ksoftirqd/321:25:013
4130920993726,8cyclictest4131629-21sshd19:24:133
4130916993727,8cyclictest30-21ksoftirqd/200:20:012
4130916993727,8cyclictest30-21ksoftirqd/200:20:012
4130916993726,8cyclictest30-21ksoftirqd/223:45:012
4130916993724,10cyclictest30-21ksoftirqd/220:35:012
4130916993724,10cyclictest1-21systemd23:50:012
413091699372,3cyclictest131rcu_preempt20:00:022
4130916993722,11cyclictest4137606-21tr21:20:012
4130916993722,11cyclictest4137606-21tr21:20:002
413091099378,17cyclictest4133423-21modprobe20:00:001
413091099374,26cyclictest4134745-21modprobe20:25:001
413091099374,26cyclictest4134745-21modprobe20:25:001
413091099373,8cyclictest23-21ksoftirqd/121:13:131
4130910993726,9cyclictest23-21ksoftirqd/119:55:011
4130910993726,9cyclictest23-21ksoftirqd/119:30:011
4130910993726,9cyclictest23-21ksoftirqd/119:30:011
4130910993726,8cyclictest23-21ksoftirqd/120:45:001
4130910993726,8cyclictest23-21ksoftirqd/120:45:001
4130910993726,7cyclictest235-21systemd-journal21:08:131
4130910993726,7cyclictest23-21ksoftirqd/122:40:011
4130910993723,11cyclictest23-21ksoftirqd/123:30:011
4130920993628,4cyclictest271-21cron21:20:013
4130920993628,4cyclictest271-21cron21:20:003
4130920993625,8cyclictest37-21ksoftirqd/319:55:013
4130920993625,7cyclictest306-21in:imuxsock23:00:023
4130920993624,9cyclictest4134732-21sshd20:23:133
4130920993624,9cyclictest4134732-21sshd20:23:133
4130920993622,10cyclictest235-21systemd-journal20:25:033
4130920993620,11cyclictest37-21ksoftirqd/300:10:023
413091699367,9cyclictest0-21swapper/200:40:022
413091699364,3cyclictest131rcu_preempt21:55:022
4130916993627,7cyclictest30-21ksoftirqd/200:35:022
413091699362,4cyclictest131rcu_preempt23:01:132
4130916993624,9cyclictest30-21ksoftirqd/220:45:012
4130916993624,9cyclictest30-21ksoftirqd/220:45:012
4130916993624,8cyclictest4132370-21sshd19:37:132
4130916993624,8cyclictest30-21ksoftirqd/220:05:022
4130916993623,10cyclictest30-21ksoftirqd/222:05:012
413091699361,3cyclictest131rcu_preempt19:45:022
413091099365,22cyclictest4135004-21munin-plugin-st20:30:001
4130910993625,7cyclictest23-21ksoftirqd/121:20:011
4130910993625,7cyclictest23-21ksoftirqd/121:20:011
4130910993624,8cyclictest4141977-21sshd22:42:131
4130910993616,15cyclictest235-21systemd-journal19:25:011
4130920993525,6cyclictest306-21in:imuxsock20:35:023
4130920993522,9cyclictest4147157-21sshd00:22:133
4130920993522,9cyclictest4140399-21sshd22:10:133
4130920993519,13cyclictest1-21systemd22:00:003
413091699354,3cyclictest131rcu_preempt22:10:002
4130916993526,7cyclictest30-21ksoftirqd/221:45:012
413091699352,4cyclictest131rcu_preempt22:55:002
4130916993524,8cyclictest4138031-21cut21:25:022
413091099355,4cyclictest131rcu_preempt21:25:011
413091099355,25cyclictest4138372-21grep21:35:001
413091099352,4cyclictest131rcu_preempt19:50:001
4130910993524,8cyclictest23-21ksoftirqd/123:15:011
4130910993523,10cyclictest23-21ksoftirqd/100:05:011
4130910993522,9cyclictest235-21systemd-journal23:45:011
4130910993521,10cyclictest235-21systemd-journal23:40:001
413092099344,4cyclictest131rcu_preempt19:10:023
4130920993426,4cyclictest1-21systemd22:50:003
4130920993425,5cyclictest306-21in:imuxsock21:15:003
4130920993423,9cyclictest37-21ksoftirqd/321:50:013
4130916993424,7cyclictest4135001-21munin-plugin-st20:30:002
4130916993424,7cyclictest30-21ksoftirqd/200:34:132
4130916993419,11cyclictest1-21systemd22:25:512
413091699341,3cyclictest131rcu_preempt23:55:032
413091099344,4cyclictest131rcu_preempt00:30:001
413091099344,3cyclictest131rcu_preempt23:45:131
4130910993426,5cyclictest4143270-21cron23:10:001
4130910993424,8cyclictest23-21ksoftirqd/123:54:131
4130910993422,6cyclictest306-21in:imuxsock22:05:001
413092099332,6cyclictest131rcu_preempt00:30:013
413092099332,3cyclictest131rcu_preempt21:05:013
4130920993323,8cyclictest37-21ksoftirqd/323:36:133
413091099334,22cyclictest235-21systemd-journal21:55:001
4130910993323,6cyclictest4142267-21systemd-run22:50:021
413091099332,26cyclictest4136826-21true21:05:011
4130910993318,11cyclictest272-21dbus-daemon23:05:001
413091099331,28cyclictest1-21systemd20:35:011
4130920993220,8cyclictest4141399-21latency_hist22:30:023
413091699321,4cyclictest131rcu_preempt19:15:022
413091099324,16cyclictest235-21systemd-journal22:30:001
4130920993125,4cyclictest4143030-21latency_hist23:05:013
4130916993122,7cyclictest30-21ksoftirqd/222:25:012
4130916993119,8cyclictest30-21ksoftirqd/219:10:022
413091699311,4cyclictest131rcu_preempt00:25:002
413092099303,4cyclictest131rcu_preempt21:55:013
413092099302,3cyclictest131rcu_preempt20:05:023
4130916992917,9cyclictest1-21systemd23:35:012
413092099283,4cyclictest131rcu_preempt23:14:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional