You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-15 - 10:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Sat Feb 15, 2025 00:44:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
254542799122113,7cyclictest37-21ksoftirqd/323:15:023
2545423999411,8cyclictest131rcu_preempt21:05:022
2545423998111,4cyclictest131rcu_preempt20:05:022
2545427996720,11cyclictest37-21ksoftirqd/300:00:023
2545427996720,11cyclictest37-21ksoftirqd/300:00:013
254542399672,14cyclictest2548745-21munin-run20:15:012
254542399662,18cyclictest2555640-21unin-run22:30:002
254542399662,13cyclictest2547681-21cron19:55:012
2545423996620,40cyclictest1-21systemd22:55:012
254542399642,15cyclictest2558397-21/usr/sbin/munin23:25:012
254542399642,14cyclictest2556153-21munin-run22:40:022
2545423996324,35cyclictest2550304-21latency_hist20:45:012
2545423996322,36cyclictest306-21in:imuxsock00:23:512
254542399632,17cyclictest2552639-21munin-run21:30:012
254542399631,14cyclictest2560922-21modprobe00:13:512
254542399629,15cyclictest272-21dbus-daemon23:55:022
254542399622,16cyclictest2546154-21munin-run19:25:002
2545423996217,5cyclictest2557508-21latency_hist23:05:022
2545423996216,41cyclictest1-21systemd00:00:012
2545423996216,41cyclictest1-21systemd00:00:012
254542399621,15cyclictest2550812-21munin-run20:55:012
2545423996141,15cyclictest1-21systemd22:20:002
254542399612,19cyclictest2545914-21munin-run19:20:012
2545423996121,34cyclictest306-21in:imuxsock00:20:002
2545423996118,39cyclictest2546640-21sshd19:34:202
254542399611,14cyclictest363-21snmpd00:32:502
2545423996110,44cyclictest0-21swapper/223:00:002
254542399602,16cyclictest2559674-21cat23:50:012
254542399592,16cyclictest2551073-21munin-run21:00:002
2545423995917,5cyclictest2546915-21/usr/sbin/munin19:40:022
2545423995915,39cyclictest306-21in:imuxsock20:25:012
254542799586,7cyclictest37-21ksoftirqd/323:05:003
2545423995820,34cyclictest30-21ksoftirqd/220:20:012
2545423995819,35cyclictest2545776-21latency_hist19:15:022
2545423995819,35cyclictest2545776-21latency_hist19:15:012
2545427995713,9cyclictest37-21ksoftirqd/322:40:013
2545427995713,10cyclictest37-21ksoftirqd/321:20:013
254542399572,16cyclictest2553405-21wc21:45:022
254542399572,13cyclictest2550573-21munin-run20:50:012
254542399571,16cyclictest2554372-21munin-run22:05:002
2545427995613,11cyclictest37-21ksoftirqd/319:15:023
2545427995613,11cyclictest37-21ksoftirqd/319:15:013
254542399564,9cyclictest2558623-21modprobe23:30:002
254542399562,17cyclictest2559113-21cron23:40:012
254542799554,12cyclictest37-21ksoftirqd/321:35:023
2545427995516,9cyclictest131rcu_preempt22:35:013
2545423995513,36cyclictest0-21swapper/219:30:012
254542399551,15cyclictest2549769-21cron20:35:002
254542399551,14cyclictest2552541-21ls21:25:012
2545413995538,12cyclictest235-21systemd-journal20:00:000
254542399542,12cyclictest271-21cron21:20:012
254542399541,17cyclictest2561746-21unin-run00:30:012
254542399541,15cyclictest2554626-21munin-run22:10:012
254541899543,7cyclictest23-21ksoftirqd/100:39:591
2545413995436,11cyclictest1-21systemd19:20:010
254542799537,9cyclictest37-21ksoftirqd/321:10:023
2545427995317,11cyclictest37-21ksoftirqd/323:40:023
2545427995316,8cyclictest37-21ksoftirqd/322:50:013
2545427995312,10cyclictest37-21ksoftirqd/319:20:013
2545423995334,13cyclictest272-21dbus-daemon19:50:002
254542399533,14cyclictest2557864-21cat23:15:012
254542399532,15cyclictest2547191-21munin-run19:45:002
2545423995319,30cyclictest2550026-21latency_hist20:40:032
2545423995318,30cyclictest2558273-21latency_hist23:20:022
254542399531,16cyclictest2554316-21latency_hist22:00:032
254542399531,15cyclictest2553620-21munin-run21:50:002
254542799525,9cyclictest37-21ksoftirqd/322:45:013
254542399522,15cyclictest2553861-21munin-run21:55:012
2545423995221,27cyclictest2548181-21sed20:00:022
2545423995213,7cyclictest2553159-21pluginstate21:40:002
2545427995135,8cyclictest0-21swapper/322:25:013
254542399512,13cyclictest2556642-21latency_hist22:50:012
254542399511,15cyclictest2556395-21unin-run22:45:002
2545418995135,12cyclictest272-21dbus-daemon23:40:001
2545418995135,11cyclictest1-21systemd23:55:001
2545418995113,5cyclictest131rcu_preempt21:35:031
2545413995136,11cyclictest235-21systemd-journal21:45:010
2545413995133,14cyclictest2557625-21latency_hist23:10:020
254542799504,6cyclictest37-21ksoftirqd/320:55:013
254542399502,17cyclictest2555385-21cut22:25:012
254542799499,11cyclictest37-21ksoftirqd/323:35:033
254542799496,13cyclictest37-21ksoftirqd/320:00:013
254542399492,16cyclictest2560691-21modprobe00:09:592
254542399492,15cyclictest2557604-21munin-run23:10:002
254542399492,15cyclictest2548656-21cat20:10:032
254542399492,15cyclictest2548656-21cat20:10:022
254542399492,10cyclictest2562245-21modprobe00:40:002
254542399491,14cyclictest2560493-21sed00:05:012
2545418994933,11cyclictest1-21systemd23:20:011
2545418994929,15cyclictest272-21dbus-daemon22:19:591
2545413994942,4cyclictest16950irq/224-800000020:20:000
254541399492,19cyclictest235-21systemd-journal21:05:010
254542799486,7cyclictest37-21ksoftirqd/323:10:023
2545427994828,14cyclictest272-21dbus-daemon00:25:013
254542799481,6cyclictest37-21ksoftirqd/319:10:013
254542399482,14cyclictest2556046-21latency_hist22:35:022
2545418994834,9cyclictest2553125-21systemctl21:40:001
2545418994818,24cyclictest131rcu_preempt20:30:021
254541399482,7cyclictest235-21systemd-journal23:00:010
254541399482,33cyclictest151rcuc/000:30:030
254542799475,12cyclictest37-21ksoftirqd/323:25:023
254542799474,11cyclictest37-21ksoftirqd/323:55:013
254542799473,9cyclictest37-21ksoftirqd/320:20:023
2545427994733,7cyclictest2546913-21sendmail19:40:003
2545427994733,10cyclictest2553137-21tr21:40:003
254542799471,11cyclictest2552764-21latency_hist21:30:023
2545423994728,14cyclictest235-21systemd-journal20:30:012
254541899479,6cyclictest131rcu_preempt20:05:001
254541899476,13cyclictest23-21ksoftirqd/121:45:011
2545418994736,8cyclictest23-21ksoftirqd/122:35:001
2545418994711,8cyclictest131rcu_preempt19:10:011
2545427994638,6cyclictest37-21ksoftirqd/322:15:003
254542799462,13cyclictest37-21ksoftirqd/320:15:013
254541899466,6cyclictest131rcu_preempt20:00:011
254541899465,6cyclictest131rcu_preempt22:50:001
2545418994635,8cyclictest221rcuc/121:55:001
2545418994632,10cyclictest235-21systemd-journal22:00:011
254541899462,9cyclictest23-21ksoftirqd/119:20:011
2545418994624,15cyclictest1-21systemd19:50:001
254541899461,7cyclictest0-21swapper/122:59:591
254541899461,10cyclictest23-21ksoftirqd/122:45:021
2545413994629,13cyclictest1-21systemd21:00:010
254542799456,10cyclictest37-21ksoftirqd/300:10:013
254542799452,8cyclictest37-21ksoftirqd/300:30:003
254542799451,7cyclictest2562292-21unin-run00:40:013
2545423994513,7cyclictest30-21ksoftirqd/221:14:522
254541899458,11cyclictest23-21ksoftirqd/122:55:011
254541899451,10cyclictest23-21ksoftirqd/123:15:001
2545413994527,13cyclictest1-21systemd00:25:010
2545413994525,3cyclictest131rcu_preempt19:50:020
2545413994524,16cyclictest272-21dbus-daemon23:20:000
254542799444,9cyclictest37-21ksoftirqd/320:40:023
254542799443,8cyclictest37-21ksoftirqd/321:02:243
254542799442,8cyclictest37-21ksoftirqd/321:55:013
254542799442,8cyclictest37-21ksoftirqd/320:30:013
2545427994428,12cyclictest235-21systemd-journal21:21:223
254542799442,11cyclictest37-21ksoftirqd/319:35:013
254542799441,8cyclictest2555117-21munin-run22:20:013
254541899446,21cyclictest221rcuc/122:25:021
2545418994437,5cyclictest23-21ksoftirqd/122:40:021
254541899442,6cyclictest131rcu_preempt20:20:011
254541899441,9cyclictest23-21ksoftirqd/100:35:011
254541399443,6cyclictest2550026-21latency_hist20:40:020
254541399442,9cyclictest2559654-21munin-run23:50:010
2545413994429,10cyclictest306-21in:imuxsock23:59:590
2545413994429,10cyclictest306-21in:imuxsock00:00:000
2545413994427,12cyclictest272-21dbus-daemon22:10:010
254541399442,12cyclictest2554111-21munin-run22:00:000
254542799435,7cyclictest37-21ksoftirqd/322:30:023
254542799434,9cyclictest37-21ksoftirqd/323:26:563
254542799434,9cyclictest37-21ksoftirqd/300:04:593
2545427994334,7cyclictest37-21ksoftirqd/320:50:013
2545427994329,10cyclictest2551193-21wc21:00:023
2545427994328,10cyclictest235-21systemd-journal23:00:003
254542799431,11cyclictest2548377-21latency_hist20:05:033
2545423994328,11cyclictest235-21systemd-journal23:40:302
254542399432,14cyclictest2551623-21munin-run21:10:012
254541899434,10cyclictest23-21ksoftirqd/120:55:021
2545418994333,7cyclictest23-21ksoftirqd/120:50:011
254541899432,7cyclictest23-21ksoftirqd/121:26:311
254541899432,7cyclictest23-21ksoftirqd/120:35:021
2545418994322,15cyclictest272-21dbus-daemon23:30:011
254541899431,8cyclictest23-21ksoftirqd/123:50:021
254541899431,7cyclictest235-21systemd-journal21:20:001
2545413994330,9cyclictest235-21systemd-journal19:29:590
2545413994327,13cyclictest12-21ksoftirqd/019:40:000
2545413994322,15cyclictest272-21dbus-daemon21:40:010
254541399431,38cyclictest2548716-21sshd20:14:230
254542799426,7cyclictest37-21ksoftirqd/322:55:023
254542799425,10cyclictest37-21ksoftirqd/300:15:013
254542799423,9cyclictest37-21ksoftirqd/321:45:013
254542799423,6cyclictest37-21ksoftirqd/319:22:213
254542399422,15cyclictest2554874-21systemd-run22:15:012
254541899421,5cyclictest0-21swapper/100:30:011
254541899421,11cyclictest23-21ksoftirqd/121:00:001
254541399425,33cyclictest2547453-21kworker/0:019:50:020
2545413994226,13cyclictest235-21systemd-journal23:23:560
2545413994219,5cyclictest12-21ksoftirqd/023:55:030
254542799416,9cyclictest37-21ksoftirqd/321:49:593
2545427994131,7cyclictest361rcuc/321:15:013
2545427994128,9cyclictest2547178-21cron19:44:593
2545423994122,14cyclictest1-21systemd21:35:022
254542399412,14cyclictest235-21systemd-journal23:34:592
254541899419,6cyclictest131rcu_preempt23:25:021
254541899415,5cyclictest131rcu_preempt22:15:021
2545413994130,6cyclictest306-21in:imuxsock22:25:000
2545413994128,6cyclictest306-21in:imuxsock20:50:010
2545413994127,11cyclictest272-21dbus-daemon00:05:000
254541399412,33cyclictest2552329-21sort21:20:020
2545413994123,12cyclictest272-21dbus-daemon23:15:010
254541399411,37cyclictest0-21swapper/020:35:020
254542799405,9cyclictest37-21ksoftirqd/320:25:013
2545427994032,6cyclictest37-21ksoftirqd/300:20:003
254542799402,7cyclictest0-21swapper/322:00:383
2545418994026,11cyclictest23-21ksoftirqd/120:10:021
2545418994026,11cyclictest23-21ksoftirqd/120:10:011
2545418994026,10cyclictest221rcuc/120:40:001
2545418994023,11cyclictest235-21systemd-journal23:59:591
2545418994023,11cyclictest235-21systemd-journal00:00:001
2545418994012,8cyclictest131rcu_preempt21:25:021
2545418994011,5cyclictest131rcu_preempt19:45:011
2545413994033,4cyclictest16950irq/224-800000022:05:010
2545413994032,5cyclictest19050irq/34-mmc122:52:220
2545413994030,6cyclictest306-21in:imuxsock21:30:010
2545413994028,9cyclictest408-21lldpd21:48:370
2545413994027,10cyclictest235-21systemd-journal22:35:350
2545413994024,11cyclictest272-21dbus-daemon20:30:000
254541399402,3cyclictest131rcu_preempt22:30:020
2545413994021,14cyclictest2550282-21sshd20:42:230
254542799395,8cyclictest37-21ksoftirqd/322:10:003
2545427993931,6cyclictest37-21ksoftirqd/322:00:003
254542799391,13cyclictest0-21swapper/320:34:063
2545418993930,6cyclictest23-21ksoftirqd/122:05:011
2545418993928,8cyclictest2555639-21systemd-run22:30:011
2545418993928,7cyclictest2561190-21cron00:20:001
2545418993917,17cyclictest221rcuc/121:01:261
2545418993915,19cyclictest221rcuc/123:35:021
2545418993913,5cyclictest131rcu_preempt19:35:011
2545418993910,6cyclictest131rcu_preempt20:23:501
254541399393,31cyclictest0-21swapper/019:33:150
2545413993927,9cyclictest2555119-21systemctl22:20:000
2545413993925,11cyclictest2548223-21latency_hist20:05:020
2545413993921,12cyclictest16950irq/224-800000021:10:000
254541399391,6cyclictest2554867-21systemd-run22:15:000
254541399391,33cyclictest2560954-21munin-run00:15:010
254542799385,7cyclictest37-21ksoftirqd/320:45:003
254542799384,9cyclictest37-21ksoftirqd/319:30:003
2545427993829,7cyclictest37-21ksoftirqd/320:10:013
2545427993829,7cyclictest37-21ksoftirqd/320:10:003
254542799382,10cyclictest37-21ksoftirqd/323:20:003
2545418993827,9cyclictest2550291-21sshd20:44:231
2545418993826,9cyclictest23-21ksoftirqd/100:15:001
254541899382,5cyclictest131rcu_preempt23:05:021
2545418993815,19cyclictest23-21ksoftirqd/121:07:021
254541899381,4cyclictest0-21swapper/119:11:101
254541899381,4cyclictest0-21swapper/119:11:101
254541399383,30cyclictest0-21swapper/023:27:180
2545413993830,4cyclictest16950irq/224-800000000:40:000
254541399382,32cyclictest2555848-21sshd22:33:160
2545413993823,10cyclictest272-21dbus-daemon19:25:010
2545413993822,11cyclictest2559029-21sed23:35:020
2545413993820,13cyclictest272-21dbus-daemon21:35:010
254541899375,9cyclictest23-21ksoftirqd/120:15:001
254541899375,6cyclictest23-21ksoftirqd/119:37:201
2545418993728,6cyclictest23-21ksoftirqd/122:10:001
2545418993724,10cyclictest23-21ksoftirqd/123:45:001
254541899371,5cyclictest131rcu_preempt00:10:021
254541399373,30cyclictest0-21swapper/019:40:260
2545413993730,4cyclictest16950irq/224-800000000:32:520
2545413993723,10cyclictest12-21ksoftirqd/021:55:020
2545413993718,13cyclictest272-21dbus-daemon00:10:010
254542799362,8cyclictest37-21ksoftirqd/323:50:013
2545427993628,5cyclictest37-21ksoftirqd/323:45:013
254541899366,20cyclictest221rcuc/119:30:021
2545418993626,7cyclictest23-21ksoftirqd/121:50:011
254541899361,7cyclictest0-21swapper/100:23:111
254541399365,21cyclictest235-21systemd-journal21:15:010
254541399363,30cyclictest0-21swapper/023:39:230
254541399363,29cyclictest0-21swapper/000:16:420
2545413993627,5cyclictest19050irq/34-mmc120:06:450
2545413993627,5cyclictest19050irq/34-mmc120:06:440
2545413993625,7cyclictest306-21in:imuxsock23:45:000
254541399361,3cyclictest131rcu_preempt19:15:000
254541399361,3cyclictest131rcu_preempt19:14:590
254541399361,31cyclictest0-21swapper/020:23:320
254542799351,8cyclictest0-21swapper/319:46:123
254541899359,5cyclictest131rcu_preempt19:53:041
254541899355,7cyclictest23-21ksoftirqd/121:15:001
2545418993510,5cyclictest131rcu_preempt19:25:001
2545413993528,4cyclictest19050irq/34-mmc120:50:260
2545413993527,5cyclictest19050irq/34-mmc121:23:100
2545427993425,5cyclictest271-21cron19:55:003
254541399342,3cyclictest131rcu_preempt22:45:020
254541399341,4cyclictest2556599-21sshd22:46:260
254541399324,20cyclictest2545415-21latency_hist19:10:000
2545413993221,8cyclictest12-21ksoftirqd/023:05:010
254542799311,5cyclictest0-21swapper/300:33:453
2545418993123,5cyclictest2560161-21latency_hist00:00:031
2545418993013,14cyclictest272-21dbus-daemon23:10:011
254542399271,7cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional