You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-16 - 13:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Tue Apr 16, 2024 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
278686994539,4cyclictest596191-21latency_hist23:19:591
278686994135,4cyclictest559264-21latency_hist22:55:001
278694993836,1cyclictest683342-21fwupd00:16:023
278689993735,1cyclictest119-1kworker/2:1H+kblockd00:16:022
278689993631,3cyclictest514778-21ssh22:25:122
278684993629,5cyclictest529288-21latency_hist22:35:000
278684993228,3cyclictest330523-21apt-get20:00:060
278689993127,3cyclictest440113-21gpgv21:35:012
278686993128,2cyclictest392237-21gpgv21:00:001
278686993126,3cyclictest397984-21apt-get21:05:131
278689993027,2cyclictest74-21kswapd021:55:002
278686993027,2cyclictest432910-21apt-get21:30:121
27868699300,29cyclictest1-21systemd22:50:011
278684993026,3cyclictest462618-21kworker/u8:0+events_unbound22:00:140
278686992926,2cyclictest633282-21kworker/u8:2+flush-259:000:00:001
278694992818,8cyclictest309453-21gpgv19:40:003
278689992824,3cyclictest74-21kswapd022:50:012
278689992823,3cyclictest314900-21latency_hist19:45:022
278689992822,4cyclictest477449-21latency_hist22:00:012
278686992824,3cyclictest430143-21kworker/u8:3+flush-259:021:55:001
278684992822,4cyclictest551472-21latency_hist22:50:000
278694992725,2cyclictest74-21kswapd022:05:013
278694992721,4cyclictest410321-21latency_hist21:15:003
278686992723,3cyclictest623952-21kworker/u8:0+flush-259:023:40:301
278686992619,5cyclictest253-21systemd-journal22:25:001
278684992620,4cyclictest625849-21latency_hist23:40:000
278686992522,2cyclictest303372-21kworker/u8:2+flush-259:019:45:001
278684992522,1cyclictest0-21swapper/021:35:010
278689992417,5cyclictest544092-21latency_hist22:45:012
278686992421,2cyclictest580813-21ssh23:08:531
278686992421,2cyclictest439206-21kworker/u8:2+flush-259:021:45:011
278686992420,3cyclictest425710-21ssh21:25:141
278686992420,3cyclictest253-21systemd-journal00:39:281
278684992420,3cyclictest303372-21kworker/u8:2+flush-259:019:30:180
278694992320,2cyclictest74-21kswapd023:24:593
278694992319,3cyclictest403109-21kworker/u8:1+flush-259:021:20:003
278689992320,2cyclictest462618-21kworker/u8:0+flush-259:022:00:162
278689992319,3cyclictest440744-21ssh21:35:122
278686992319,3cyclictest662861-21latency_hist00:05:011
278684992320,2cyclictest303372-21kworker/u8:2+flush-259:019:35:240
278694992218,3cyclictest492129-21ssh22:09:483
278694992217,3cyclictest392320-21gpgv21:00:003
278689992216,4cyclictest707754-21latency_hist00:35:012
278686992218,3cyclictest512349-21kworker/u8:2+flush-259:022:35:191
278686992218,3cyclictest335591-21/usr/sbin/munin20:05:141
278684992219,2cyclictest526895-21kworker/u8:3+flush-259:022:55:200
278684992219,2cyclictest309380-21kworker/u8:3+flush-259:020:10:000
278684992217,3cyclictest309899-21gpgv19:40:010
278694992118,2cyclictest288849-21latency_hist19:20:013
278694992117,3cyclictest74-21kswapd022:20:013
27869499211,2cyclictest715167-21apt00:40:003
278686992118,2cyclictest74-21kswapd022:30:011
278684992118,2cyclictest403109-21kworker/u8:1+events_unbound21:10:290
278684992117,3cyclictest74-21kswapd023:30:000
278684992115,4cyclictest581956-21diskstats23:10:140
278694992016,3cyclictest608076-21ssh23:25:243
278689992015,3cyclictest566316-21ssh22:59:242
278686992015,4cyclictest253-21systemd-journal22:00:011
278684992016,3cyclictest239704-21kworker/u8:2+events_unbound19:15:140
278684992016,3cyclictest239704-21kworker/u8:2+events_unbound19:10:090
278684992014,4cyclictest356147-21/usr/sbin/munin20:25:200
278694991916,2cyclictest521833-21cron22:30:003
278694991915,3cyclictest664130-21irqcore00:05:193
278694991915,3cyclictest431945-21ssh21:28:523
278694991913,4cyclictest618764-21apt-get23:35:113
278694991913,4cyclictest371955-21apt-get20:40:113
278689991916,2cyclictest455224-21cron21:45:002
278689991915,2cyclictest544115-21kworker/u8:1+events_unbound23:15:002
278689991914,3cyclictest512349-21kworker/u8:2+flush-259:022:35:142
278686991916,2cyclictest633282-21kworker/u8:2+flush-259:000:19:571
278686991915,3cyclictest371385-21gpgv20:40:011
278686991913,5cyclictest74-21kswapd022:05:141
278684991915,3cyclictest500037-21apt-get22:15:140
278684991915,2cyclictest603590-21dump-pmu-power23:25:000
278684991915,2cyclictest526895-21kworker/u8:3+flush-259:000:30:200
278684991913,4cyclictest345626-21latency_hist20:15:000
278689991815,2cyclictest600594-21needreboot23:20:212
278689991814,3cyclictest74-21kswapd023:00:132
278689991813,4cyclictest403212-21apt-get21:10:132
278689991813,3cyclictest552082-21cat22:50:012
278686991815,2cyclictest526895-21kworker/u8:3+flush-259:000:25:341
278684991815,2cyclictest74-21kswapd023:44:590
278684991815,2cyclictest670702-21idleruntime-cro00:10:000
278684991815,2cyclictest526895-21kworker/u8:3+events_unbound00:16:030
278684991815,2cyclictest403109-21kworker/u8:1+flush-259:021:27:120
278684991815,2cyclictest278679-21cyclictest20:35:000
278684991814,3cyclictest633282-21kworker/u8:2+events_unbound00:27:290
278684991814,3cyclictest61150irq/132-enp0s29f1:rx-023:30:140
278684991814,2cyclictest0-21swapper/022:00:010
278684991813,2cyclictest61150irq/132-enp0s29f1:rx-021:44:030
278684991811,5cyclictest371929-21/usr/sbin/munin20:40:130
278694991714,2cyclictest526895-21kworker/u8:3+flush-259:022:40:593
278694991714,2cyclictest439988-21idleruntime-cro21:35:003
278694991713,3cyclictest74-21kswapd021:25:013
278694991713,2cyclictest676921-21ssh00:12:473
278694991713,2cyclictest491ktimers/323:30:213
278694991711,4cyclictest278867-21/usr/sbin/munin19:10:103
278694991711,4cyclictest253-21systemd-journal23:55:003
27869499170,14cyclictest470076-21munin-run21:55:013
278689991714,2cyclictest514-21in:imuxsock20:29:592
278689991713,3cyclictest417340-21ssh21:19:132
278689991712,3cyclictest253-21systemd-journal20:10:012
278684991714,2cyclictest355906-21idleruntime-cro20:24:590
278684991714,2cyclictest320058-21kworker/u8:0+flush-259:021:35:270
278684991714,2cyclictest320058-21kworker/u8:0+events_unbound21:18:290
278684991714,2cyclictest278668-21kworker/u8:3+events_unbound19:20:190
278684991713,3cyclictest708032-21kworker/u8:0+flush-259:000:35:240
278684991713,2cyclictest61150irq/132-enp0s29f1:rx-022:05:140
278684991713,2cyclictest455603-21apt-get21:45:130
278684991712,3cyclictest544115-21kworker/u8:1+events_unbound23:15:270
278684991712,1cyclictest61150irq/132-enp0s29f1:rx-023:00:160
278694991613,2cyclictest74-21kswapd023:15:003
278694991612,3cyclictest567003-21apt-get23:00:123
278694991612,2cyclictest330252-21cron20:00:013
278694991610,4cyclictest361043-21latency_hist20:30:013
278689991614,1cyclictest0-21swapper/200:40:002
278689991613,2cyclictest74-21kswapd023:25:132
278689991613,2cyclictest401ktimers/223:31:512
278689991612,3cyclictest633282-21kworker/u8:2+flush-259:000:12:382
278689991612,2cyclictest579989-21ssh23:07:212
278689991611,3cyclictest626188-21/usr/sbin/munin23:40:182
278689991611,3cyclictest507019-21munin-run22:20:012
27868999160,14cyclictest1-21systemd20:25:002
278686991613,2cyclictest74-21kswapd023:50:001
278686991613,2cyclictest705819-21ssh00:31:261
278686991613,2cyclictest430143-21kworker/u8:3+flush-259:021:45:231
27868499168,7cyclictest514765-21apt-get22:25:110
278684991614,1cyclictest61150irq/132-enp0s29f1:rx-023:07:310
278684991614,1cyclictest61150irq/132-enp0s29f1:rx-022:50:160
278684991614,1cyclictest61150irq/132-enp0s29f1:rx-020:40:000
278684991614,1cyclictest61150irq/132-enp0s29f1:rx-000:00:220
278684991613,2cyclictest61150irq/132-enp0s29f1:rx-022:20:210
278684991613,2cyclictest61150irq/132-enp0s29f1:rx-021:50:260
278684991613,2cyclictest61150irq/132-enp0s29f1:rx-021:22:280
278684991612,3cyclictest633282-21kworker/u8:2+events_unbound23:47:440
278684991612,3cyclictest309380-21kworker/u8:3+events_unbound20:15:150
27869499159,4cyclictest580791-21ssh23:08:503
278694991510,3cyclictest320058-21kworker/u8:0+flush-259:021:35:203
278689991512,2cyclictest494732-21kthreadcore22:10:182
278689991512,2cyclictest392638-21gpgconf21:00:002
278689991512,1cyclictest0-21swapper/220:14:592
278689991510,3cyclictest0-21swapper/221:30:002
278686991512,2cyclictest74-21kswapd000:25:001
278686991512,2cyclictest566647-21munin-run23:00:001
278686991512,2cyclictest407840-21unixbench_singl21:10:271
278686991511,3cyclictest609656-21ssh23:27:281
278686991511,3cyclictest572239-21ssh23:01:401
278686991510,3cyclictest253-21systemd-journal20:42:001
27868499158,5cyclictest330272-21latency_hist20:00:010
278684991513,2cyclictest61150irq/132-enp0s29f1:rx-023:55:280
278684991513,1cyclictest61150irq/132-enp0s29f1:rx-023:51:280
278684991513,1cyclictest61150irq/132-enp0s29f1:rx-022:42:020
278684991513,1cyclictest61150irq/132-enp0s29f1:rx-022:36:270
278684991513,1cyclictest61150irq/132-enp0s29f1:rx-019:50:150
278684991512,2cyclictest61150irq/132-enp0s29f1:rx-021:05:140
278684991512,2cyclictest61150irq/132-enp0s29f1:rx-020:55:270
278684991512,2cyclictest171ktimers/000:20:040
278684991511,2cyclictest381930-21turbostat20:50:000
27869499149,3cyclictest455547-21ssh21:45:103
27869499148,4cyclictest6987021ssh00:27:073
27869499147,4cyclictest253-21systemd-journal19:25:003
27869499141,5cyclictest320006-21awk19:50:003
278694991410,3cyclictest632821-21ssh23:44:143
278694991410,3cyclictest535214-21kworker/u8:0+events_unbound22:45:203
278694991410,3cyclictest462618-21kworker/u8:0+flush-259:021:55:283
27868999148,5cyclictest0-21swapper/219:15:012
278689991411,2cyclictest662855-21cron00:05:002
278689991411,2cyclictest508-21systemd-logind00:25:002
278689991410,3cyclictest490443-21ssh22:06:362
27868699149,4cyclictest253-21systemd-journal22:20:011
27868699149,4cyclictest253-21systemd-journal20:02:271
278686991411,2cyclictest675233-21ntp_kernel_pll_00:10:261
278686991411,2cyclictest320058-21kworker/u8:0+flush-259:021:21:321
27868699141,12cyclictest1-21systemd19:50:001
278686991410,3cyclictest663333-21cat00:05:011
278684991413,1cyclictest61150irq/132-enp0s29f1:rx-000:10:030
278684991411,2cyclictest61150irq/132-enp0s29f1:rx-020:50:140
278684991411,2cyclictest309380-21kworker/u8:3+events_unbound19:45:180
27869499139,3cyclictest497311-21ssh22:10:363
27869499139,2cyclictest516-21rs:main3
278694991311,1cyclictest0-21swapper/320:49:593
278694991310,2cyclictest74-21kswapd023:15:123
278694991310,2cyclictest534590-21ssh22:36:043
278694991310,2cyclictest491ktimers/322:55:553
278694991310,2cyclictest491ktimers/300:01:033
27868999139,3cyclictest424149-21ssh21:23:032
278689991311,1cyclictest0-21swapper/220:35:192
278689991310,2cyclictest693435-21cut00:25:142
278689991310,2cyclictest459627-21needreboot21:45:242
278689991310,2cyclictest299763-21irqcore19:30:102
27868699139,3cyclictest483490-21ssh22:02:251
27868699138,4cyclictest309380-21kworker/u8:3+flush-259:020:15:241
27868699136,5cyclictest644867-21cut23:50:221
278686991310,2cyclictest544115-21kworker/u8:1+events_unbound23:10:201
278686991310,2cyclictest439984-21apt-get21:35:021
278684991311,1cyclictest61150irq/132-enp0s29f1:rx-019:25:060
27869499129,2cyclictest700695-21cpuspeed_turbos00:30:143
27869499129,1cyclictest0-21swapper/321:43:553
27869499128,3cyclictest529088-21ssh22:34:403
27869499121,3cyclictest397723-21latency_hist21:05:003
278694991210,1cyclictest655448-21turbostat00:00:003
27868999129,2cyclictest527239-21ssh22:31:132
27868999129,2cyclictest294351-21cpuspeed_turbos19:25:092
27868999129,2cyclictest0-21swapper/220:00:012
27868999129,2cyclictest0-21swapper/219:20:002
278689991210,1cyclictest0-21swapper/221:10:002
27868699129,2cyclictest618845-21cpuspeed_turbos23:35:131
27868699129,2cyclictest615901-21unixbench_multi23:30:261
27868699129,2cyclictest522296-21cpuspeed_turbos22:30:111
27868699129,2cyclictest407904-21kworker/u8:3+flush-259:021:15:221
27868699129,1cyclictest0-21swapper/120:20:201
278686991210,1cyclictest393911-21kthreadcore21:00:191
278686991210,1cyclictest0-21swapper/120:30:211
278686991210,1cyclictest0-21swapper/120:10:281
27868699120,5cyclictest309350-21apt-get19:40:011
27868499128,3cyclictest2904-21cron21:05:000
27869499119,1cyclictest0-21swapper/319:35:003
27869499118,2cyclictest491ktimers/322:20:133
27869499118,2cyclictest345957-21cpuspeed_turbos20:15:123
27869499118,2cyclictest0-21swapper/320:10:003
27869499117,2cyclictest0-21swapper/320:55:003
27869499116,3cyclictest552167-21ssh22:50:063
27869499116,3cyclictest253-21systemd-journal20:31:273
27869499115,4cyclictest402524-21sshd21:06:293
27869499115,4cyclictest278820-21cat19:10:003
27869499110,2cyclictest343027-21kthreadcore20:10:223
27868999119,1cyclictest0-21swapper/223:55:002
27868999119,1cyclictest0-21swapper/223:45:132
27868999118,2cyclictest74-21kswapd023:40:002
27868999118,2cyclictest401ktimers/223:15:262
27868999118,2cyclictest0-21swapper/219:25:002
27868999114,5cyclictest309355-21https19:40:002
27868999111,3cyclictest507378-21apt-get22:20:122
27868699119,2cyclictest239704-21kworker/u8:2+flush-259:019:10:111
27868699119,1cyclictest0-21swapper/123:20:181
27868699118,2cyclictest74-21kswapd022:40:141
27868699118,2cyclictest360562-21unixbench_multi20:25:241
27868699118,2cyclictest0-21swapper/119:30:081
27868499111,2cyclictest278617-21latency_hist19:10:010
27869499107,2cyclictest3305212sleep320:00:053
27869499107,2cyclictest0-21swapper/300:20:173
27869499107,1cyclictest0-21swapper/320:40:013
27869499106,3cyclictest278668-21kworker/u8:3+flush-259:019:21:453
27869499106,2cyclictest3514142sleep320:20:183
27868999108,1cyclictest324429-21sensors19:50:182
27868999106,2cyclictest345894-21seq20:15:122
27868999106,2cyclictest0-21swapper/220:00:132
27868999105,4cyclictest650878-21cut23:55:192
27868999100,9cyclictest670774-21cat00:10:002
27868699108,1cyclictest320640-21mii-tool19:50:111
27868699108,1cyclictest298381-21sensors19:25:201
27868699108,1cyclictest288126-21sensors19:15:191
27868699107,2cyclictest330269-21sh20:00:001
27868699106,3cyclictest497578-21ssh22:11:071
2786949996,2cyclictest324743-21users19:50:193
2786949996,1cyclictest0-21swapper/319:40:083
2786899997,1cyclictest0-21swapper/220:55:012
2786899997,1cyclictest0-21swapper/220:50:002
2786899997,1cyclictest0-21swapper/219:50:002
2786899996,2cyclictest74-21kswapd020:40:132
2786899996,1cyclictest365574-21timerwakeupswit20:30:272
2786869997,1cyclictest381918-21idleruntime-cro20:50:001
2786869996,2cyclictest0-21swapper/119:25:001
2786869995,2cyclictest0-21swapper/120:50:231
2786899971,4cyclictest393192-21hddtemp_smartct21:00:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional