You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-19 - 04:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Fri Jul 19, 2024 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
92821994133,5cyclictest255-21systemd-journal21:35:000
92821994035,3cyclictest255-21systemd-journal20:55:000
92827993933,2cyclictest132697-21kworker/u8:1+events_unbound19:45:251
92827993734,2cyclictest484095-21kworker/u8:0+events_unbound00:30:121
92827993632,3cyclictest155495-21kworker/u8:3+events_unbound21:25:221
92821993632,2cyclictest155495-21kworker/u8:3+events_unbound20:25:010
92827993531,3cyclictest132697-21kworker/u8:1+flush-259:020:05:011
92836993431,2cyclictest305553-21kworker/u8:4+events_unbound22:08:023
92827993430,2cyclictest132697-21kworker/u8:1+events_unbound20:20:191
92827993327,4cyclictest509534-21latency_hist00:15:001
92831993227,4cyclictest336861-21kworker/u8:2+flush-259:022:35:002
92827993228,3cyclictest368616-21ssh22:45:121
92827993128,2cyclictest352463-21kworker/u8:1+flush-259:022:50:181
92827993128,2cyclictest118249-21kworker/u8:3+events_unbound19:55:191
92827993124,2cyclictest224153-21kworker/u8:2+events_unbound21:45:171
92827993124,2cyclictest224153-21kworker/u8:2+events_unbound21:45:171
92836993028,2cyclictest186275-21kworker/u8:1+events_unbound20:48:003
92827993027,2cyclictest274120-21kworker/u8:3+events_unbound22:15:241
92827993027,2cyclictest224153-21kworker/u8:2+flush-259:021:30:201
92836992927,2cyclictest186275-21kworker/u8:1+events_unbound20:43:003
92836992927,1cyclictest186275-21kworker/u8:1+events_unbound21:19:553
92831992925,3cyclictest186275-21kworker/u8:1+flush-259:021:00:142
92827992926,2cyclictest242728-21kworker/u8:0+events_unbound22:00:111
92827992825,2cyclictest428696-21kworker/u8:2+flush-259:023:50:191
92827992825,2cyclictest155495-21kworker/u8:3+flush-259:020:34:301
92827992824,3cyclictest274120-21kworker/u8:3+events_unbound21:50:181
92836992725,2cyclictest274120-21kworker/u8:3+events_unbound22:30:033
92836992724,2cyclictest383901-21kworker/u8:0+events_unbound23:02:003
92827992724,2cyclictest398149-21kworker/u8:1+flush-259:023:06:091
92827992723,3cyclictest61506-21kworker/u8:1+events_unbound19:10:241
92827992723,3cyclictest420488-21kworker/u8:4+events_unbound23:20:191
92821992721,4cyclictest470048-21apt-get23:50:010
92821992721,4cyclictest431356-21/usr/sbin/munin23:25:110
92836992624,2cyclictest224153-21kworker/u8:2+events_unbound21:59:003
92836992624,2cyclictest224153-21kworker/u8:2+events_unbound21:12:003
92836992624,2cyclictest186275-21kworker/u8:1+events_unbound21:04:323
92836992624,2cyclictest132697-21kworker/u8:1+events_unbound20:21:023
92836992624,1cyclictest274120-21kworker/u8:3+events_unbound22:58:123
92836992622,3cyclictest202831-21gpgv20:55:003
92836992621,3cyclictest255-21systemd-journal00:22:093
92827992623,2cyclictest484095-21kworker/u8:0+flush-259:000:05:121
92827992623,2cyclictest428696-21kworker/u8:2+flush-259:023:30:151
92827992623,2cyclictest420488-21kworker/u8:4+events_unbound23:19:001
92827992623,2cyclictest186275-21kworker/u8:1+events_unbound20:40:191
92827992621,2cyclictest132697-21kworker/u8:1+flush-259:020:15:011
92827992620,2cyclictest509587-21kworker/u8:1+flush-259:000:20:201
92821992622,3cyclictest509587-21kworker/u8:1+flush-259:000:20:190
92836992523,2cyclictest274120-21kworker/u8:3+events_unbound22:29:363
92836992522,2cyclictest478307-21cron23:55:003
92836992522,2cyclictest321216-21kworker/u8:1+events_unbound22:20:563
92831992522,2cyclictest242728-21kworker/u8:0+flush-259:022:15:002
92827992522,2cyclictest274120-21kworker/u8:3+flush-259:022:05:211
92827992522,2cyclictest118249-21kworker/u8:3+flush-259:019:45:011
92827992521,3cyclictest255-21systemd-journal22:20:591
92827992520,4cyclictest238043-21ssh21:20:171
92821992522,2cyclictest186275-21kworker/u8:1+flush-259:021:15:470
92821992521,2cyclictest118249-21kworker/u8:3+flush-259:020:00:130
92836992422,2cyclictest155495-21kworker/u8:3+events_unbound21:33:503
92836992422,1cyclictest242728-21kworker/u8:0+events_unbound21:51:143
92836992421,2cyclictest540895-21kworker/u8:2+events_unbound00:38:383
92836992421,2cyclictest274120-21kworker/u8:3+events_unbound21:48:433
92836992421,2cyclictest274120-21kworker/u8:3+events_unbound21:48:433
92836992421,2cyclictest242728-21kworker/u8:0+events_unbound22:12:023
92836992421,2cyclictest144229-21kworker/u8:0+events_unbound20:10:003
92831992419,4cyclictest485288-21ssh23:58:382
92827992421,2cyclictest398149-21kworker/u8:1+flush-259:023:10:191
92827992420,3cyclictest438905-21kworker/u8:3+flush-259:023:55:211
92827992420,3cyclictest438905-21kworker/u8:3+flush-259:023:35:211
92827992420,3cyclictest321216-21kworker/u8:1+flush-259:022:25:061
92821992420,3cyclictest140093-21apt-get19:55:100
92836992321,2cyclictest438905-21kworker/u8:3+events_unbound23:47:483
92836992321,2cyclictest398149-21kworker/u8:1+events_unbound23:09:193
92836992320,2cyclictest428696-21kworker/u8:2+events_unbound23:36:503
92836992320,2cyclictest428696-21kworker/u8:2+events_unbound23:28:143
92836992320,2cyclictest352463-21kworker/u8:1+events_unbound22:41:303
92836992320,2cyclictest336861-21kworker/u8:2+events_unbound22:36:283
92836992320,2cyclictest224153-21kworker/u8:2+events_unbound21:29:323
92831992319,3cyclictest72197-21kworker/u8:2+flush-259:019:30:152
92827992320,2cyclictest509587-21kworker/u8:1+flush-259:000:25:201
92827992320,2cyclictest186275-21kworker/u8:1+events_unbound21:05:001
92827992318,4cyclictest350331-21ssh22:31:311
92821992320,2cyclictest321216-21kworker/u8:1+flush-259:022:20:000
92821992319,3cyclictest171335-21apt-get20:25:090
92821992317,4cyclictest185783-21ntpq20:35:210
92821992316,5cyclictest255-21systemd-journal21:10:000
92836992220,1cyclictest438905-21kworker/u8:3+events_unbound23:31:503
92836992219,2cyclictest407579-21kworker/u8:0+events_unbound23:12:473
92836992219,2cyclictest352463-21kworker/u8:1+events_unbound22:51:333
92831992219,2cyclictest274120-21kworker/u8:3+events_unbound22:05:192
92831992218,3cyclictest274120-21kworker/u8:3+flush-259:022:20:182
92831992216,5cyclictest415372-21latency_hist23:15:012
92831992216,4cyclictest423899-21ssh23:20:142
92831992216,4cyclictest255-21systemd-journal23:53:592
92831992210,1cyclictest0-21swapper/200:05:002
92827992219,2cyclictest484095-21kworker/u8:0+flush-259:000:39:421
92821992218,3cyclictest61506-21kworker/u8:1+events_unbound19:10:260
92821992218,3cyclictest376223-21apt-get22:50:100
92821992212,8cyclictest415663-21kworker/u8:3+flush-259:023:15:190
92836992118,2cyclictest509587-21kworker/u8:1+events_unbound00:16:293
92836992118,2cyclictest493977-21kworker/u8:3+events_unbound00:34:163
92836992118,2cyclictest224153-21kworker/u8:2+events_unbound21:43:103
92836992116,3cyclictest112544-21ntpq19:25:173
92831992116,3cyclictest213777-21/usr/sbin/munin21:05:132
92827992118,2cyclictest493977-21kworker/u8:3+flush-259:000:16:481
92827992116,3cyclictest134881-21/usr/sbin/munin19:50:191
92827992114,2cyclictest155495-21kworker/u8:3+events_unbound21:05:171
92821992118,2cyclictest509587-21kworker/u8:1+events_unbound00:15:210
92836992018,2cyclictest428696-21kworker/u8:2+events_unbound23:22:583
92836992017,2cyclictest155495-21kworker/u8:3+events_unbound21:09:553
92831992015,3cyclictest383652-21latency_hist22:55:012
92827992017,2cyclictest454456-21kworker/u8:1+events_unbound23:48:001
92827992017,2cyclictest438905-21kworker/u8:3+events_unbound23:44:361
92827992017,2cyclictest375925-21kworker/u8:2+events_unbound22:56:191
92827992017,2cyclictest224153-21kworker/u8:2+flush-259:021:58:141
92827992017,2cyclictest103153-21kworker/u8:0+flush-259:019:30:211
92827992017,2cyclictest103153-21kworker/u8:0+events_unbound19:20:131
92821992018,1cyclictest274120-21kworker/u8:3+events_unbound21:50:220
92821992016,3cyclictest336861-21kworker/u8:2+events_unbound22:25:200
92821992014,4cyclictest155448-21latency_hist20:10:010
92836991916,2cyclictest438905-21kworker/u8:3+events_unbound23:40:283
92836991916,2cyclictest155495-21kworker/u8:3+events_unbound20:39:443
92836991916,2cyclictest155495-21kworker/u8:3+events_unbound20:34:383
92831991915,3cyclictest186275-21kworker/u8:1+flush-259:020:40:202
92831991915,2cyclictest365624-21kworker/u8:0+flush-259:022:45:202
92831991914,4cyclictest452217-21ssh23:36:202
92827991917,1cyclictest352463-21kworker/u8:1+flush-259:022:41:221
92827991916,2cyclictest242728-21kworker/u8:0+events_unbound22:12:081
92827991916,2cyclictest224153-21kworker/u8:2+flush-259:021:37:141
92827991916,2cyclictest103153-21kworker/u8:0+flush-259:019:25:121
92821991916,2cyclictest144229-21kworker/u8:0+flush-259:020:20:010
92821991914,4cyclictest484095-21kworker/u8:0+flush-259:000:10:210
92836991816,1cyclictest428696-21kworker/u8:2+events_unbound00:10:083
92836991816,1cyclictest144229-21kworker/u8:0+events_unbound20:00:243
92836991815,2cyclictest484095-21kworker/u8:0+events_unbound00:26:083
92836991815,2cyclictest224153-21kworker/u8:2+events_unbound21:39:173
92836991814,3cyclictest352463-21kworker/u8:1+flush-259:022:48:003
92836991813,3cyclictest420736-21ssh23:15:543
92831991815,2cyclictest198184-21irqcore20:50:142
92831991814,3cyclictest390652-21ssh22:58:382
92831991814,2cyclictest274120-21kworker/u8:3+flush-259:021:55:172
92831991813,3cyclictest160632-21cron20:15:012
92831991812,4cyclictest255-21systemd-journal00:35:002
92827991816,1cyclictest383901-21kworker/u8:0+flush-259:023:01:351
92827991815,2cyclictest454456-21kworker/u8:1+events_unbound00:00:191
92827991815,2cyclictest224153-21kworker/u8:2+events_unbound21:40:571
92827991815,2cyclictest224153-21kworker/u8:2+events_unbound21:16:581
92827991815,2cyclictest224153-21kworker/u8:2+events_unbound21:12:221
92827991815,2cyclictest144229-21kworker/u8:0+events_unbound20:26:001
92827991815,2cyclictest132697-21kworker/u8:1+flush-259:020:15:211
92827991814,3cyclictest155495-21kworker/u8:3+events_unbound20:51:021
92827991813,2cyclictest352463-21kworker/u8:1+events_unbound22:39:331
92821991814,3cyclictest454456-21kworker/u8:1+flush-259:023:58:470
92821991811,5cyclictest274093-21latency_hist21:45:010
92821991811,5cyclictest119158-21cat19:35:010
92836991715,2cyclictest46554-21kworker/u8:3+events_unbound19:13:193
92836991715,2cyclictest454456-21kworker/u8:1+events_unbound23:55:473
92836991715,2cyclictest428696-21kworker/u8:2+events_unbound00:03:133
92836991715,2cyclictest124298-21kworker/u8:2+events_unbound19:48:343
92836991715,1cyclictest118249-21kworker/u8:3+events_unbound19:34:273
92836991714,2cyclictest61506-21kworker/u8:1+events_unbound19:18:143
92836991714,2cyclictest484095-21kworker/u8:0+events_unbound00:07:253
92836991714,2cyclictest297881-21kworker/u8:1+events_unbound22:01:593
92836991714,2cyclictest186275-21kworker/u8:1+flush-259:021:20:033
92836991714,2cyclictest144229-21kworker/u8:0+events_unbound20:28:093
92836991714,2cyclictest144229-21kworker/u8:0+events_unbound20:19:353
92836991714,2cyclictest132697-21kworker/u8:1+events_unbound20:10:593
92836991714,2cyclictest118249-21kworker/u8:3+events_unbound19:39:563
92836991714,2cyclictest103153-21kworker/u8:0+events_unbound19:22:543
92831991713,3cyclictest438905-21kworker/u8:3+flush-259:023:40:312
92821991714,2cyclictest274120-21kworker/u8:3+flush-259:022:24:460
92821991713,3cyclictest438905-21kworker/u8:3+events_unbound23:40:200
92821991713,3cyclictest420488-21kworker/u8:4+flush-259:023:20:200
92821991712,4cyclictest192417-21base6420:45:010
92821991711,4cyclictest129465-21latency_hist19:45:010
9283699163,2cyclictest50-21ksoftirqd/319:55:013
92836991613,2cyclictest155495-21kworker/u8:3+events_unbound20:56:123
92831991613,2cyclictest258430-21latency_hist21:35:002
92831991613,2cyclictest0-21swapper/221:15:002
92831991612,3cyclictest499109-21ssh00:05:362
92831991612,3cyclictest144229-21kworker/u8:0+flush-259:020:33:042
92827991613,2cyclictest428696-21kworker/u8:2+flush-259:023:25:151
92821991614,1cyclictest60650irq/128-enp0s29f1:rx-023:54:430
92821991614,1cyclictest60650irq/128-enp0s29f1:rx-023:35:100
92821991614,1cyclictest60650irq/128-enp0s29f1:rx-023:09:490
92821991614,1cyclictest60650irq/128-enp0s29f1:rx-022:32:160
92821991614,1cyclictest60650irq/128-enp0s29f1:rx-021:27:380
92821991614,1cyclictest60650irq/128-enp0s29f1:rx-000:01:110
92821991613,2cyclictest60650irq/128-enp0s29f1:rx-022:49:580
92821991613,2cyclictest60650irq/128-enp0s29f1:rx-022:35:180
92821991613,2cyclictest60650irq/128-enp0s29f1:rx-022:00:420
92821991613,2cyclictest60650irq/128-enp0s29f1:rx-021:10:200
92821991613,2cyclictest60650irq/128-enp0s29f1:rx-000:40:000
92821991613,2cyclictest60650irq/128-enp0s29f1:rx-000:05:160
92821991612,3cyclictest508-21in:imuxsock20:35:010
92821991612,3cyclictest124298-21kworker/u8:2+flush-259:019:50:010
92821991610,5cyclictest102281-21ntpq19:15:200
92821991610,4cyclictest274630-21diskstats21:45:130
92821991610,4cyclictest274630-21diskstats21:45:130
92836991512,2cyclictest118249-21kworker/u8:3+events_unbound19:40:483
9283199159,3cyclictest255-21systemd-journal22:19:552
92831991512,2cyclictest336861-21kworker/u8:2+flush-259:022:35:342
92831991512,2cyclictest255-21systemd-journal20:50:002
92831991511,3cyclictest399938-21rred23:05:012
92831991510,4cyclictest342058-21ssh22:25:402
92831991510,3cyclictest399549-21store23:05:012
92827991513,1cyclictest0-21swapper/119:40:011
92821991514,1cyclictest61450irq/129-enp0s29f1:tx-000:30:050
92821991513,1cyclictest61450irq/129-enp0s29f1:tx-022:43:100
92821991513,1cyclictest60650irq/128-enp0s29f1:rx-023:30:160
92821991513,1cyclictest60650irq/128-enp0s29f1:rx-023:00:380
92821991513,1cyclictest60650irq/128-enp0s29f1:rx-022:11:100
92821991513,1cyclictest60650irq/128-enp0s29f1:rx-022:05:360
92821991513,1cyclictest60650irq/128-enp0s29f1:rx-000:25:220
92821991512,2cyclictest60650irq/128-enp0s29f1:rx-021:35:130
92821991511,3cyclictest60650irq/128-enp0s29f1:rx-022:55:110
92821991511,3cyclictest398149-21kworker/u8:1+events_unbound23:10:140
92821991510,2cyclictest211rcuc/020:40:160
92831991413,1cyclictest401ktimers/223:17:242
92831991412,1cyclictest0-21swapper/223:35:012
92831991412,1cyclictest0-21swapper/221:20:012
92831991410,3cyclictest407579-21kworker/u8:0+flush-259:023:25:102
92831991410,3cyclictest304736-21ssh22:03:462
92831991410,3cyclictest224153-21kworker/u8:2+flush-259:021:45:272
92831991410,3cyclictest224153-21kworker/u8:2+flush-259:021:45:272
92821991412,1cyclictest60650irq/128-enp0s29f1:rx-020:58:480
92821991410,3cyclictest132697-21kworker/u8:1+flush-259:019:55:010
9283199139,3cyclictest46554-21kworker/u8:3+flush-259:019:11:042
9283199138,3cyclictest186275-21kworker/u8:1+events_unbound20:40:012
92831991311,1cyclictest510974-21kthreadcore00:15:162
92831991311,1cyclictest401ktimers/200:35:542
92831991311,1cyclictest0-21swapper/219:40:012
92831991311,1cyclictest0-21swapper/200:20:232
92831991310,2cyclictest462691-21cpuspeed_turbos23:45:122
92831991310,2cyclictest274120-21kworker/u8:3+flush-259:021:50:272
92831991310,2cyclictest260414-21kthreadcore21:35:162
92831991310,1cyclictest0-21swapper/222:40:252
9282199139,3cyclictest144229-21kworker/u8:0+flush-259:020:10:160
9283199129,2cyclictest401ktimers/200:26:372
9283199129,2cyclictest266734-21cpuspeed_turbos21:40:132
9283199129,2cyclictest247690-21unixbench_multi21:25:252
9283199128,3cyclictest72197-21kworker/u8:2+flush-259:019:20:212
9283199127,4cyclictest255-21systemd-journal21:22:592
92831991210,1cyclictest0-21swapper/219:40:252
9283199117,3cyclictest46554-21kworker/u8:3+flush-259:019:15:162
9282799118,3cyclictest61506-21kworker/u8:1+events_unbound19:15:081
9283199108,1cyclictest140228-21cpuspeed_turbos19:55:122
9283199107,2cyclictest151438-21irqcore20:05:152
9283199107,2cyclictest145533-21fschecks_time20:00:112
928319997,1cyclictest0-21swapper/220:15:142
928319996,2cyclictest170643-21unixbench_singl20:20:282
928319996,2cyclictest129899-21fschecks_count19:45:172
928319995,3cyclictest118249-21kworker/u8:3+flush-259:019:50:372
928319994,3cyclictest509-21rs:main2
928319981,5cyclictest208339-21apt21:00:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional