You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-03-01 - 18:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Mar 01, 2024 12:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20903429915066,81cyclictest12-21ksoftirqd/008:15:030
2090352991034,4cyclictest131rcu_preempt09:10:022
2090342998884,2cyclictest0-21swapper/012:00:020
2090342998884,2cyclictest0-21swapper/012:00:010
2090342998480,2cyclictest0-21swapper/007:45:030
2090347997661,10cyclictest2090349-21latency_hist07:10:011
2090347997661,10cyclictest2090349-21latency_hist07:10:001
2090356997229,11cyclictest37-21ksoftirqd/310:15:013
209035299722,16cyclictest2102534-21munin-run11:05:012
2090352997029,36cyclictest684-21sendmail-mta09:34:522
209035299687,8cyclictest2105320-21sshd11:55:362
209035299687,8cyclictest2105320-21sshd11:55:352
209035299683,15cyclictest2091100-21munin-run07:25:012
2090347996831,6cyclictest23-21ksoftirqd/108:50:031
2090347996724,11cyclictest274-21dbus-daemon08:25:011
209034299654,12cyclictest131rcu_preempt10:55:010
209035699646,14cyclictest37-21ksoftirqd/310:30:003
209035299642,15cyclictest2104039-21sshd11:31:352
209035299642,15cyclictest2091315-21sshd07:28:362
209035299642,14cyclictest2092637-21munin-run07:55:022
209035299633,13cyclictest2100427-21sshd10:24:362
2090352996323,37cyclictest30-21ksoftirqd/207:50:002
209035299631,14cyclictest2094676-21sshd08:32:352
209035299631,13cyclictest366-21snmpd10:13:412
209034299631,58cyclictest0-21swapper/011:45:030
209034299631,58cyclictest0-21swapper/011:45:030
209035299627,8cyclictest2100681-21sshd10:29:352
209035299623,8cyclictest2094439-21cron08:30:012
209035299622,15cyclictest2095488-21munin-run08:50:002
209035299622,13cyclictest2090543-21munin-run07:15:012
209035299613,15cyclictest2105103-21munin-run11:55:002
2090352996124,32cyclictest304-21in:imuxsock12:15:012
2090352996124,32cyclictest304-21in:imuxsock12:15:002
209035299612,15cyclictest2100965-21munin-run10:35:002
209035299612,12cyclictest2099098-21sshd09:58:352
2090347996128,10cyclictest235-21systemd-journal09:20:001
209035699606,8cyclictest37-21ksoftirqd/307:40:013
2090352996025,31cyclictest273-21cron11:25:012
209035299602,15cyclictest2093659-21munin-run08:15:002
209035299601,14cyclictest366-21snmpd09:03:282
2090352995948,6cyclictest2097592-21kworker/2:011:45:022
209035299593,7cyclictest235-21systemd-journal11:19:362
209035299592,15cyclictest2100296-21grep10:20:022
209035299592,14cyclictest2093907-21cron08:20:002
2090352995913,41cyclictest0-21swapper/211:49:492
2090352995913,41cyclictest0-21swapper/211:49:482
209035299591,13cyclictest2092887-21munin-run08:00:012
209035299591,13cyclictest2092887-21munin-run08:00:002
209035299591,12cyclictest366-21snmpd08:09:512
209035299591,12cyclictest2101187-21sshd10:39:352
209034299591,3cyclictest0-21swapper/009:30:020
209035299582,13cyclictest48-21kcompactd010:55:152
209035299581,13cyclictest366-21snmpd12:08:322
209034299586,48cyclictest0-21swapper/011:05:020
209035699575,9cyclictest37-21ksoftirqd/307:20:003
209035299572,9cyclictest2107424-21cron12:40:002
209035299572,16cyclictest2098378-21munin-run09:45:022
209035299572,16cyclictest2098378-21munin-run09:45:012
209035299571,11cyclictest2106661-21sshd12:23:362
209035299571,11cyclictest2091553-21sshd07:30:162
209035699566,9cyclictest37-21ksoftirqd/311:50:013
209035699566,9cyclictest37-21ksoftirqd/311:50:013
2090352995636,16cyclictest30-21ksoftirqd/211:15:022
2090352995636,16cyclictest30-21ksoftirqd/211:15:012
2090352995628,24cyclictest2094200-21/usr/sbin/munin08:25:002
209035299562,13cyclictest366-21snmpd09:23:502
209035299562,12cyclictest2105635-21munin-run12:04:592
2090352995613,38cyclictest0-21swapper/208:55:012
209035299561,12cyclictest366-21snmpd11:37:372
209035299561,11cyclictest366-21snmpd07:39:232
2090347995637,15cyclictest1-21systemd07:50:011
209034299562,8cyclictest274-21dbus-daemon08:30:010
209035299553,17cyclictest235-21systemd-journal08:45:012
209035299551,12cyclictest366-21snmpd12:30:292
209035299551,10cyclictest366-21snmpd09:19:472
209035299551,10cyclictest2098059-21sshd09:37:352
2090347995538,13cyclictest274-21dbus-daemon10:20:001
2090342995551,2cyclictest0-21swapper/012:25:010
209035699548,9cyclictest37-21ksoftirqd/311:20:003
2090356995435,12cyclictest2105341-21cron12:00:003
2090356995435,12cyclictest2105341-21cron11:59:593
209035299541,12cyclictest366-21snmpd12:17:112
209035299534,9cyclictest0-21swapper/208:40:002
209035299532,9cyclictest304-21in:imuxsock10:49:592
209035299532,19cyclictest2103816-21grep11:30:002
2090352995316,32cyclictest0-21swapper/210:10:002
209035299531,48cyclictest48-21kcompactd011:06:552
209034799535,23cyclictest2097002-21kworker/u8:309:45:031
209034299531,29cyclictest2099683-21latency_hist10:10:020
209035299522,14cyclictest2101491-21systemd10:45:012
2090352995218,29cyclictest2096794-21true09:15:012
2090352995215,34cyclictest30-21ksoftirqd/212:25:352
2090352995215,34cyclictest30-21ksoftirqd/212:25:342
209035299521,11cyclictest366-21snmpd09:53:182
209035299521,11cyclictest2097544-21sshd09:25:352
209035699515,8cyclictest37-21ksoftirqd/310:25:023
209035299511,13cyclictest307-21rs:main2
209035299511,13cyclictest235-21systemd-journal07:45:012
209035699505,10cyclictest37-21ksoftirqd/311:40:013
2090352995039,8cyclictest0-21swapper/207:10:012
2090352995039,8cyclictest0-21swapper/207:10:002
209035299501,44cyclictest48-21kcompactd010:51:232
2090352995011,35cyclictest2099421-21grep10:05:022
209035299501,11cyclictest2093163-21true08:05:012
2090352995010,4cyclictest0-21swapper/207:17:132
2090347994935,9cyclictest1-21systemd08:15:001
2090347994930,15cyclictest1-21systemd10:35:011
2090342994941,4cyclictest16850irq/224-800000011:40:010
209034299492,9cyclictest235-21systemd-journal09:10:460
209035699484,6cyclictest37-21ksoftirqd/309:35:013
2090356994838,8cyclictest37-21ksoftirqd/311:45:003
2090356994832,11cyclictest37-21ksoftirqd/310:55:003
209035699482,13cyclictest37-21ksoftirqd/308:19:593
2090356994812,11cyclictest37-21ksoftirqd/309:20:003
209035699481,12cyclictest2107186-21modprobe12:35:003
2090342994828,15cyclictest235-21systemd-journal12:15:010
2090342994828,15cyclictest235-21systemd-journal12:15:000
2090347994730,12cyclictest235-21systemd-journal08:20:001
2090342994735,8cyclictest12-21ksoftirqd/010:05:010
2090342994731,9cyclictest2093905-21cron08:19:590
2090342994729,12cyclictest12-21ksoftirqd/009:00:010
209035699467,10cyclictest37-21ksoftirqd/309:00:013
2090356994629,12cyclictest274-21dbus-daemon07:35:003
2090356994616,21cyclictest361rcuc/308:35:363
2090352994626,14cyclictest235-21systemd-journal09:50:002
2090347994632,10cyclictest235-21systemd-journal09:45:011
2090347994632,10cyclictest235-21systemd-journal09:45:001
2090342994632,11cyclictest12-21ksoftirqd/008:55:000
2090342994631,11cyclictest1-21systemd11:55:000
2090342994620,22cyclictest2092951-21latency_hist08:00:020
2090342994620,22cyclictest2092951-21latency_hist08:00:010
209035699455,7cyclictest37-21ksoftirqd/311:10:013
209035699455,10cyclictest37-21ksoftirqd/311:15:013
209035699455,10cyclictest37-21ksoftirqd/311:15:013
2090356994524,15cyclictest274-21dbus-daemon09:45:013
2090356994524,15cyclictest274-21dbus-daemon09:45:003
2090356994516,21cyclictest361rcuc/312:21:213
2090347994533,9cyclictest23-21ksoftirqd/110:05:021
2090347994528,12cyclictest235-21systemd-journal11:40:001
209034799451,3cyclictest131rcu_preempt12:20:001
209034799451,3cyclictest131rcu_preempt09:40:011
2090342994528,12cyclictest2103271-21cron11:19:590
2090342994524,16cyclictest2090792-21cron07:20:000
209035699447,7cyclictest37-21ksoftirqd/310:45:023
209035699447,11cyclictest37-21ksoftirqd/311:35:013
209035699447,11cyclictest37-21ksoftirqd/309:25:013
209035699446,7cyclictest37-21ksoftirqd/309:05:023
209035699445,10cyclictest37-21ksoftirqd/311:29:593
2090356994426,12cyclictest274-21dbus-daemon11:05:013
209035699441,7cyclictest2107485-21latency_hist12:40:003
2090342994427,3cyclictest21009151sshd10:30:350
209035699436,7cyclictest37-21ksoftirqd/307:52:363
209035699435,6cyclictest37-21ksoftirqd/308:45:263
2090356994333,7cyclictest37-21ksoftirqd/308:55:003
2090356994329,10cyclictest37-21ksoftirqd/310:20:003
209035699431,9cyclictest37-21ksoftirqd/308:34:593
2090347994328,9cyclictest23-21ksoftirqd/111:05:011
2090347994327,12cyclictest274-21dbus-daemon11:55:011
2090342994327,12cyclictest2105611-21sshd12:02:350
209034299431,5cyclictest2103567-21cut11:25:020
209035699424,8cyclictest37-21ksoftirqd/311:25:033
209035699423,7cyclictest149-21pr/ttyS208:25:543
2090356994230,8cyclictest37-21ksoftirqd/309:10:013
209035699422,7cyclictest37-21ksoftirqd/310:10:023
2090356994210,6cyclictest131rcu_preempt12:30:013
2090356994210,6cyclictest131rcu_preempt12:30:003
2090347994227,11cyclictest1-21systemd09:05:011
2090342994237,3cyclictest0-21swapper/008:05:020
2090342994231,7cyclictest2094957-21sh08:40:000
2090342994228,10cyclictest304-21in:imuxsock10:20:000
2090342994227,10cyclictest304-21in:imuxsock08:35:000
2090342994216,22cyclictest2098661-21latency_hist09:50:000
209035699415,14cyclictest37-21ksoftirqd/307:12:003
2090356994128,9cyclictest1-21systemd08:05:013
209035699411,5cyclictest0-21swapper/312:10:533
209035699411,5cyclictest0-21swapper/312:10:523
209034799413,3cyclictest131rcu_preempt11:30:001
2090347994125,11cyclictest235-21systemd-journal10:30:011
209034299415,33cyclictest2106945-21unin-run12:30:010
209034299415,33cyclictest2106945-21unin-run12:30:010
2090342994129,8cyclictest2102768-21sh11:09:590
2090342994128,9cyclictest12-21ksoftirqd/009:20:010
2090342994128,10cyclictest12-21ksoftirqd/008:48:350
209034299412,7cyclictest151rcuc/012:10:020
2090342994126,12cyclictest2106384-21sshd12:19:350
2090342994125,12cyclictest2096486-21sshd09:05:350
2090342994123,13cyclictest12-21ksoftirqd/012:40:000
2090356994029,6cyclictest2099106-21cron10:00:013
2090356994013,5cyclictest131rcu_preempt12:10:013
2090356994011,6cyclictest131rcu_preempt07:25:023
209035699401,14cyclictest37-21ksoftirqd/312:20:013
2090347994025,11cyclictest2092900-21tr08:00:011
2090347994025,11cyclictest2092900-21tr08:00:011
2090347994024,11cyclictest235-21systemd-journal07:30:031
2090347994023,11cyclictest235-21systemd-journal11:20:011
2090342994026,10cyclictest0-21swapper/007:30:000
2090342994025,12cyclictest2094162-21sshd08:20:290
2090342994025,11cyclictest2101189-21sshd10:39:350
2090342994025,11cyclictest12-21ksoftirqd/007:37:010
2090342994023,13cyclictest2107150-21sshd12:30:350
2090342994023,12cyclictest274-21dbus-daemon10:00:010
209035699399,6cyclictest131rcu_preempt11:00:023
209035699396,10cyclictest37-21ksoftirqd/308:25:013
2090356993928,7cyclictest37-21ksoftirqd/310:50:013
2090356993927,9cyclictest274-21dbus-daemon09:30:013
2090356993919,16cyclictest274-21dbus-daemon10:05:003
2090356993912,6cyclictest131rcu_preempt10:40:013
2090347993927,6cyclictest304-21in:imuxsock07:15:001
2090347993924,10cyclictest304-21in:imuxsock12:00:031
2090347993924,10cyclictest2101191-21cron10:40:001
209034799391,7cyclictest2102081-21latency_hist10:55:021
209034299398,10cyclictest131rcu_preempt07:45:010
2090342993930,5cyclictest308-21systemd-logind11:30:000
209034299392,8cyclictest2104621-21cat11:45:000
2090342993928,8cyclictest2102208-21sshd10:56:030
209034299392,5cyclictest235-21systemd-journal10:42:490
2090342993924,10cyclictest0-21swapper/008:45:020
2090342993923,12cyclictest2104035-21sshd11:31:100
209035699389,5cyclictest131rcu_preempt08:00:013
209035699389,5cyclictest131rcu_preempt08:00:003
209035699388,8cyclictest37-21ksoftirqd/307:30:013
209035699384,8cyclictest37-21ksoftirqd/309:15:013
2090356993825,10cyclictest37-21ksoftirqd/308:45:003
2090356993820,13cyclictest1-21systemd09:55:013
2090356993812,5cyclictest131rcu_preempt08:15:033
2090347993826,9cyclictest2094930-21sshd08:37:361
2090347993826,8cyclictest2091807-21sshd07:35:351
2090347993824,10cyclictest235-21systemd-journal12:25:011
2090347993823,11cyclictest1-21systemd07:30:001
2090342993832,4cyclictest0-21swapper/007:35:000
2090342993830,6cyclictest18650irq/34-mmc109:50:020
209034299382,5cyclictest131rcu_preempt07:10:010
209034299382,5cyclictest131rcu_preempt07:10:000
209034299382,32cyclictest2098112-21modprobe09:40:000
209035699378,6cyclictest131rcu_preempt09:40:013
2090356993725,9cyclictest37-21ksoftirqd/312:05:013
209035699371,7cyclictest0-21swapper/311:53:473
2090356993710,6cyclictest131rcu_preempt07:45:013
2090347993727,8cyclictest23-21ksoftirqd/108:35:021
2090347993727,7cyclictest23-21ksoftirqd/108:55:001
2090347993724,6cyclictest2102239-21sh11:00:011
2090342993724,10cyclictest12-21ksoftirqd/010:21:010
2090342993722,11cyclictest401-21lldpd07:23:040
209034299371,32cyclictest0-21swapper/010:12:060
2090342993711,5cyclictest131rcu_preempt10:48:360
209035699365,5cyclictest131rcu_preempt07:10:013
209035699365,5cyclictest131rcu_preempt07:10:003
209035699361,6cyclictest149-21pr/ttyS208:07:323
209035699361,6cyclictest0-21swapper/309:46:343
2090347993627,5cyclictest2103122-21latency_hist11:15:021
2090347993627,5cyclictest2103122-21latency_hist11:15:011
2090347993626,7cyclictest2097566-21cron09:30:011
2090347993625,9cyclictest23-21ksoftirqd/111:10:021
2090347993624,5cyclictest304-21in:imuxsock11:45:001
2090347993623,9cyclictest2092602-21sshd07:53:351
2090347993620,13cyclictest2106116-21sshd12:12:361
2090347993620,13cyclictest2106116-21sshd12:12:361
209034299362,6cyclictest235-21systemd-journal07:13:160
2090342993620,13cyclictest366-21snmpd09:43:540
2090342993620,13cyclictest366-21snmpd09:43:530
2090342993617,15cyclictest2097797-21sshd09:30:360
209035699353,9cyclictest37-21ksoftirqd/307:50:023
209034799354,3cyclictest131rcu_preempt08:05:011
2090347993527,4cyclictest2103561-21latency_hist11:25:011
2090347993526,5cyclictest2107173-21cron12:34:591
2090347993526,5cyclictest2090812-21munin-plugin-st07:20:011
2090347993524,9cyclictest23-21ksoftirqd/107:45:031
2090347993524,8cyclictest2095209-21sshd08:42:341
2090347993520,11cyclictest235-21systemd-journal08:10:021
2090347993520,11cyclictest2095977-21cron09:00:001
2090347993518,12cyclictest23-21ksoftirqd/109:09:591
209034299351,32cyclictest0-21swapper/009:20:030
209034299351,31cyclictest0-21swapper/010:25:060
209034299351,31cyclictest0-21swapper/009:01:380
209035699341,6cyclictest149-21pr/ttyS210:34:493
209034799343,4cyclictest131rcu_preempt11:35:001
2090347993427,4cyclictest2106928-21latency_hist12:30:031
2090347993427,4cyclictest2106928-21latency_hist12:30:031
2090347993423,8cyclictest23-21ksoftirqd/110:50:021
2090347993422,8cyclictest2094421-21sshd08:26:351
209034799341,3cyclictest131rcu_preempt12:00:031
209034799341,29cyclictest48-21kcompactd012:35:101
2090342993423,6cyclictest2103200-21latency_hist11:15:030
2090342993423,6cyclictest2103200-21latency_hist11:15:030
2090347993324,5cyclictest2105929-21latency_hist12:10:011
2090347993323,8cyclictest23-21ksoftirqd/110:10:011
2090347993322,8cyclictest304-21in:imuxsock12:00:011
2090347993318,11cyclictest1-21systemd09:35:011
2090347993314,14cyclictest274-21dbus-daemon10:45:011
2090342993326,4cyclictest0-21swapper/008:05:270
209034299331,29cyclictest0-21swapper/007:52:560
2090347993224,4cyclictest2098831-21sshd09:50:361
209034799322,3cyclictest131rcu_preempt10:13:481
2090347993222,8cyclictest23-21ksoftirqd/110:21:341
2090347993123,6cyclictest23-21ksoftirqd/107:25:021
2090347993121,8cyclictest23-21ksoftirqd/109:20:471
2090347993121,7cyclictest23-21ksoftirqd/109:56:551
209034799302,4cyclictest131rcu_preempt11:50:011
209034799302,4cyclictest131rcu_preempt11:50:001
2090347993017,10cyclictest0-21swapper/109:15:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional