You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-16 - 14:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Mon Jul 15, 2024 12:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3826550995651,3cyclictest3947129-21lspci09:05:121
3826558994846,1cyclictest3967229-21fwupd09:18:413
3826554994538,5cyclictest3847178-21latency_hist07:30:002
3826558994441,2cyclictest26405-21kworker/u8:3+flush-259:012:10:123
3826558994339,3cyclictest74-21kswapd012:10:003
3826558994339,3cyclictest4014251-21kworker/u8:3+events_unbound10:04:453
3826550994337,4cyclictest4045461-21ssh10:10:141
3826554994236,4cyclictest3067417-21systemd-journal10:39:572
3826546994239,2cyclictest4029658-21kworker/u8:0+flush-259:010:45:000
3826554994135,4cyclictest3067417-21systemd-journal10:15:002
3826558993934,2cyclictest3919099-21kworker/u8:0+events_unbound09:06:323
3826554993936,2cyclictest74-21kswapd009:18:432
3826546993732,4cyclictest3982700-21kworker/u8:2+events_unbound09:45:010
3826558993330,2cyclictest26405-21kworker/u8:3+events_unbound12:20:013
3826558993330,2cyclictest26405-21kworker/u8:3+events_unbound12:20:003
3826554993228,2cyclictest3919099-21kworker/u8:0+events_unbound09:26:462
3826554993223,4cyclictest4045032-21latency_hist10:10:012
3826550993227,3cyclictest74-21kswapd011:10:011
3826554993128,2cyclictest74-21kswapd011:44:592
3826550993128,2cyclictest74-21kswapd009:18:441
3826558993026,3cyclictest3982700-21kworker/u8:2+events_unbound09:30:323
3826554993028,2cyclictest3968089-21kworker/u8:4+events_unbound09:50:472
3826550993027,2cyclictest74-21kswapd010:05:001
3826558992925,3cyclictest60240-21kworker/u8:0+flush-259:012:30:273
3826558992925,3cyclictest4115054-21kworker/u8:1+events_unbound11:22:093
3826546992922,4cyclictest3946724-21cat09:05:120
3826558992824,3cyclictest3919099-21kworker/u8:0+events_unbound09:25:233
3826558992822,3cyclictest4115054-21kworker/u8:1+events_unbound11:26:323
3826558992821,5cyclictest3899401-21latency_hist08:20:013
3826554992825,2cyclictest4029658-21kworker/u8:0+events_unbound10:30:382
3826554992825,2cyclictest3968089-21kworker/u8:4+events_unbound09:48:462
3826554992825,2cyclictest3967323-21kworker/u8:1+events_unbound09:20:532
3826546992823,3cyclictest74-21kswapd011:45:000
3826546992822,4cyclictest4114779-21latency_hist10:55:020
3826558992724,2cyclictest3788829-21kworker/u8:0+events_unbound07:10:283
3826558992723,3cyclictest3847401-21kworker/u8:3+flush-259:008:10:333
3826558992722,3cyclictest74-21kswapd010:25:113
3826558992719,2cyclictest3919099-21kworker/u8:0+events_unbound08:40:143
3826554992724,2cyclictest3982700-21kworker/u8:2+events_unbound09:39:462
3826554992724,2cyclictest3982700-21kworker/u8:2+events_unbound09:34:452
3826554992724,2cyclictest3858171-21kworker/u8:4+events_unbound07:50:172
3826554992723,2cyclictest4128154-21kworker/u8:2+events_unbound11:02:452
3826550992722,3cyclictest3919100-21kworker/u8:4+flush-259:008:35:361
3826546992723,3cyclictest74-21kswapd009:18:440
3826558992622,2cyclictest3999138-21kworker/u8:1+events_unbound09:40:303
3826554992624,2cyclictest3951191-21kworker/u8:3+events_unbound09:07:072
3826554992624,2cyclictest3857957-21kworker/u8:2+events_unbound08:00:012
3826554992623,2cyclictest44849-21kworker/u8:4+events_unbound12:23:462
3826554992623,2cyclictest44849-21kworker/u8:4+events_unbound12:23:462
3826554992623,2cyclictest3847401-21kworker/u8:3+flush-259:008:05:152
3826554992623,2cyclictest3793783-21kworker/u8:1+events_unbound08:04:432
3826554992623,2cyclictest3793783-21kworker/u8:1+events_unbound07:35:282
3826554992618,2cyclictest26405-21kworker/u8:3+events_unbound12:19:412
3826554992618,2cyclictest26405-21kworker/u8:3+events_unbound12:19:402
3826546992620,4cyclictest3884290-21diskstats08:05:150
3826554992522,2cyclictest4115054-21kworker/u8:1+events_unbound11:38:532
3826554992522,2cyclictest4045068-21kworker/u8:1+events_unbound10:18:472
3826554992522,2cyclictest4029658-21kworker/u8:0+events_unbound10:42:462
3826554992522,2cyclictest3968089-21kworker/u8:4+events_unbound09:55:582
3826554992516,3cyclictest4014251-21kworker/u8:3+events_unbound10:03:372
3826550992518,5cyclictest3889072-21latency_hist08:10:011
3826546992521,3cyclictest3793783-21kworker/u8:1+flush-259:007:35:000
3826558992421,2cyclictest4167991-21kworker/u8:2+events_unbound11:55:243
3826558992420,3cyclictest74-21kswapd010:40:123
3826558992420,3cyclictest3858171-21kworker/u8:4+events_unbound08:00:253
3826558992418,4cyclictest516-21rs:main3
3826558992417,5cyclictest83737-21latency_hist12:40:003
3826554992421,2cyclictest4128154-21kworker/u8:2+events_unbound11:07:462
3826554992421,2cyclictest3951191-21kworker/u8:3+events_unbound09:10:382
3826554992418,2cyclictest4105232-21kworker/u8:3+events_unbound11:19:582
3826550992421,2cyclictest3858171-21kworker/u8:4+flush-259:007:50:251
3826546992419,3cyclictest3935542-21sh08:55:000
3826558992319,3cyclictest3919100-21kworker/u8:4+flush-259:008:50:223
3826558992318,4cyclictest3975487-21ssh09:24:363
3826558992318,3cyclictest3067417-21systemd-journal11:50:003
3826558992311,7cyclictest3925461-21kworker/u8:1+events_unbound08:55:133
3826554992321,1cyclictest4105232-21kworker/u8:3+events_unbound11:27:032
3826554992320,2cyclictest4029658-21kworker/u8:0+events_unbound10:56:482
3826554992320,2cyclictest3982700-21kworker/u8:2+events_unbound09:41:472
3826554992319,2cyclictest60240-21kworker/u8:0+events_unbound12:30:012
3826554992316,2cyclictest3919099-21kworker/u8:0+events_unbound08:45:232
3826550992320,2cyclictest3793783-21kworker/u8:1+events_unbound08:15:251
3826546992316,5cyclictest4083890-21latency_hist10:35:010
3826558992219,2cyclictest4152941-21kworker/u8:0+events_unbound11:37:123
3826558992219,2cyclictest4115054-21kworker/u8:1+flush-259:011:18:403
3826558992219,2cyclictest4105232-21kworker/u8:3+flush-259:011:15:013
3826558992219,2cyclictest4029658-21kworker/u8:0+events_unbound10:22:533
3826558992219,2cyclictest4029658-21kworker/u8:0+events_unbound10:10:293
3826558992219,2cyclictest3894251-21kworker/u8:0+flush-259:008:20:333
3826558992217,3cyclictest22081-21/usr/sbin/munin12:00:333
3826554992220,2cyclictest3830795-21kworker/u8:2+events_unbound07:20:192
3826554992220,1cyclictest4152941-21kworker/u8:0+events_unbound11:58:442
3826554992219,2cyclictest60240-21kworker/u8:0+events_unbound12:36:442
3826554992219,2cyclictest4167991-21kworker/u8:2+flush-259:011:50:072
3826554992219,2cyclictest4115054-21kworker/u8:1+events_unbound11:45:142
3826554992219,2cyclictest3793783-21kworker/u8:1+events_unbound07:30:312
3826554992218,2cyclictest3904165-21kworker/u8:2+events_unbound08:30:382
3826554992217,2cyclictest3919099-21kworker/u8:0+events_unbound09:00:222
3826550992219,2cyclictest3847401-21kworker/u8:3+flush-259:008:10:371
3826550992219,2cyclictest2904-21cron10:25:001
3826550992217,3cyclictest3983435-21ls09:30:001
3826550992216,4cyclictest4076525-21/usr/sbin/munin10:30:361
3826546992219,2cyclictest4128154-21kworker/u8:2+events_unbound11:05:000
3826546992218,3cyclictest4167991-21kworker/u8:2+flush-259:011:55:300
3826558992118,2cyclictest4115054-21kworker/u8:1+events_unbound10:59:463
3826558992118,2cyclictest4029658-21kworker/u8:0+events_unbound10:31:493
3826558992118,2cyclictest4014251-21kworker/u8:3+events_unbound09:50:493
3826558992118,2cyclictest3904165-21kworker/u8:2+events_unbound08:30:213
3826558992118,2cyclictest3793783-21kworker/u8:1+events_unbound07:30:133
3826558992117,3cyclictest3968089-21kworker/u8:4+events_unbound09:56:383
382655499218,3cyclictest41-21ksoftirqd/208:55:002
3826554992119,2cyclictest26405-21kworker/u8:3+events_unbound12:04:512
3826554992118,2cyclictest4115054-21kworker/u8:1+events_unbound11:34:492
3826554992118,2cyclictest4115054-21kworker/u8:1+events_unbound11:13:532
3826554992118,2cyclictest4076467-21kworker/u8:2+events_unbound10:48:472
3826554992118,2cyclictest4059100-21kworker/u8:3+events_unbound10:24:572
3826554992118,2cyclictest3830795-21kworker/u8:2+events_unbound07:15:282
3826554992116,2cyclictest26405-21kworker/u8:3+events_unbound12:12:122
3826550992117,3cyclictest74-21kswapd010:35:311
3826546992118,2cyclictest4029658-21kworker/u8:0+events_unbound10:10:270
3826546992117,3cyclictest3967323-21kworker/u8:1+events_unbound09:20:250
3826546992117,3cyclictest3847401-21kworker/u8:3+flush-259:007:45:330
3826558992017,2cyclictest4105232-21kworker/u8:3+flush-259:011:06:263
3826558992017,2cyclictest3847401-21kworker/u8:3+events_unbound08:05:183
3826558992017,2cyclictest26405-21kworker/u8:3+events_unbound12:21:213
3826558992017,2cyclictest26405-21kworker/u8:3+events_unbound12:21:213
3826558992016,3cyclictest4045068-21kworker/u8:1+events_unbound10:15:253
3826554992017,2cyclictest60240-21kworker/u8:0+events_unbound12:27:262
3826554992017,2cyclictest4076467-21kworker/u8:2+events_unbound10:50:222
3826554992017,2cyclictest3894251-21kworker/u8:0+events_unbound08:20:372
3826554992017,2cyclictest3847401-21kworker/u8:3+events_unbound07:41:462
3826550992015,4cyclictest42998-21kworker/u8:2+flush-259:012:20:171
3826550992015,4cyclictest42998-21kworker/u8:2+flush-259:012:20:161
3826550992015,3cyclictest14596-21ssh11:55:141
3826550992013,5cyclictest3067417-21systemd-journal11:15:001
3826546992015,4cyclictest4177263-21latency_hist11:35:000
3826558991916,2cyclictest4076467-21kworker/u8:2+events_unbound10:53:093
3826558991916,2cyclictest4029658-21kworker/u8:0+flush-259:011:00:143
3826558991916,2cyclictest4029658-21kworker/u8:0+flush-259:010:45:403
3826558991916,2cyclictest3982700-21kworker/u8:2+events_unbound10:05:303
3826558991916,2cyclictest3836868-21kworker/u8:0+flush-259:007:37:563
3826558991915,3cyclictest3968089-21kworker/u8:4+flush-259:009:46:013
3826558991913,3cyclictest4152941-21kworker/u8:0+events_unbound11:41:073
3826550991916,2cyclictest74-21kswapd010:00:001
3826550991916,1cyclictest0-21swapper/112:20:011
3826550991916,1cyclictest0-21swapper/112:20:001
3826550991915,3cyclictest26405-21kworker/u8:3+flush-259:012:25:311
3826546991915,3cyclictest60240-21kworker/u8:0+flush-259:012:30:190
3826558991815,2cyclictest4167991-21kworker/u8:2+flush-259:011:30:063
3826558991815,2cyclictest4115054-21kworker/u8:1+events_unbound11:50:363
3826558991814,3cyclictest3951191-21kworker/u8:3+flush-259:009:10:463
3826558991814,3cyclictest3858171-21kworker/u8:4+events_unbound07:43:443
3826554991816,2cyclictest3847401-21kworker/u8:3+events_unbound08:13:272
3826554991815,2cyclictest3919100-21kworker/u8:4+events_unbound08:43:462
3826554991815,2cyclictest3904165-21kworker/u8:2+events_unbound08:35:262
3826550991815,2cyclictest4108527-21kthreadcore10:50:211
3826550991814,3cyclictest74-21kswapd012:15:001
3826550991814,3cyclictest3968089-21kworker/u8:4+flush-259:009:50:281
3826550991812,4cyclictest4130159-21kworker/u8:4+flush-259:011:15:311
3826546991814,2cyclictest61150irq/132-enp0s29f1:rx-011:20:230
3826558991714,2cyclictest60240-21kworker/u8:0+events_unbound12:26:443
3826558991714,2cyclictest4029658-21kworker/u8:0+events_unbound10:35:453
3826558991714,2cyclictest3857957-21kworker/u8:2+flush-259:007:55:263
3826558991713,3cyclictest3793783-21kworker/u8:1+events_unbound07:15:203
382655499175,6cyclictest0-21swapper/207:45:172
3826550991714,2cyclictest74-21kswapd010:50:001
3826550991713,3cyclictest3830795-21kworker/u8:2+flush-259:007:15:001
382655099171,15cyclictest3998872-21idleruntime-cro09:40:001
3826550991711,4cyclictest516-21rs:main1
3826546991714,2cyclictest61950irq/133-enp0s29f1:tx-010:45:300
3826546991714,2cyclictest61150irq/132-enp0s29f1:rx-009:50:260
3826546991714,2cyclictest171ktimers/010:28:000
3826546991713,3cyclictest3936207-21kworker/u8:3+flush-259:008:55:200
3826546991713,3cyclictest3858171-21kworker/u8:4+flush-259:008:00:270
3826546991713,3cyclictest3847401-21kworker/u8:3+flush-259:008:10:260
3826546991713,3cyclictest171ktimers/010:56:580
3826546991713,2cyclictest61150irq/132-enp0s29f1:rx-012:15:080
3826546991713,2cyclictest61150irq/132-enp0s29f1:rx-012:15:070
3826546991713,2cyclictest61150irq/132-enp0s29f1:rx-012:10:000
3826546991713,2cyclictest1-21systemd07:15:010
3826546991712,4cyclictest3953002-21apt-get09:10:140
3826554991613,2cyclictest3811117-21kworker/u8:3+flush-259:007:14:102
3826550991613,2cyclictest80675-21ntp_offset12:35:321
3826550991613,2cyclictest3905827-21kthreadcore08:25:221
3826550991613,1cyclictest0-21swapper/111:30:011
3826550991612,3cyclictest4053454-21/usr/sbin/munin10:15:351
3826550991612,3cyclictest3793783-21kworker/u8:1+events_unbound07:35:161
3826550991611,4cyclictest26780-21ssh12:00:341
3826550991611,4cyclictest0-21swapper/107:50:011
3826550991611,3cyclictest3935516-21kworker/u8:2+events_unbound09:10:281
3826546991614,2cyclictest61150irq/132-enp0s29f1:rx-009:35:220
3826546991614,1cyclictest61150irq/132-enp0s29f1:rx-011:38:130
3826546991614,1cyclictest61150irq/132-enp0s29f1:rx-011:15:170
3826546991614,1cyclictest61150irq/132-enp0s29f1:rx-011:08:080
3826546991614,1cyclictest61150irq/132-enp0s29f1:rx-007:55:210
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-012:22:510
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-012:22:510
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-012:11:480
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-010:17:500
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-010:05:200
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-009:56:540
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-009:45:260
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-008:40:280
3826546991613,2cyclictest61150irq/132-enp0s29f1:rx-008:25:270
3826546991613,2cyclictest3967323-21kworker/u8:1+flush-259:009:25:150
3826546991613,2cyclictest171ktimers/011:12:020
3826546991613,2cyclictest171ktimers/010:35:320
3826546991613,2cyclictest171ktimers/010:02:100
3826546991612,3cyclictest514-21in:imuxsock12:40:000
3826546991612,3cyclictest3847401-21kworker/u8:3+flush-259:007:45:010
3826546991612,2cyclictest61150irq/132-enp0s29f1:rx-012:00:110
3826558991513,1cyclictest3847165-21cron07:29:593
3826558991512,2cyclictest3919099-21kworker/u8:0+flush-259:009:00:253
3826558991512,1cyclictest0-21swapper/308:39:593
3826554991512,2cyclictest3936207-21kworker/u8:3+events_unbound08:59:532
3826554991512,2cyclictest3793783-21kworker/u8:1+events_unbound08:19:182
3826550991513,1cyclictest311ktimers/112:08:361
3826550991512,2cyclictest74-21kswapd010:55:151
3826550991512,2cyclictest3999138-21kworker/u8:1+flush-259:009:45:001
3826546991514,1cyclictest61950irq/133-enp0s29f1:tx-011:45:270
3826546991513,1cyclictest61150irq/132-enp0s29f1:rx-012:25:050
3826546991513,1cyclictest171ktimers/011:29:100
3826546991512,2cyclictest61150irq/132-enp0s29f1:rx-011:50:270
3826546991512,2cyclictest61150irq/132-enp0s29f1:rx-010:22:020
3826546991512,2cyclictest61150irq/132-enp0s29f1:rx-009:30:290
3826546991511,3cyclictest3793783-21kworker/u8:1+flush-259:007:25:210
3826558991411,2cyclictest3836868-21kworker/u8:0+events_unbound07:23:463
3826550991411,2cyclictest3974719-21ssh09:23:151
3826550991410,3cyclictest0-21swapper/108:34:591
3826546991412,2cyclictest61150irq/132-enp0s29f1:rx-007:20:120
3826546991412,1cyclictest61150irq/132-enp0s29f1:rx-009:05:000
3826546991411,2cyclictest3793783-21kworker/u8:1+flush-259:008:15:210
3826558991311,1cyclictest0-21swapper/307:55:013
3826554991310,2cyclictest3905002-21fschecks_count08:25:172
382655099139,3cyclictest508-21systemd-logind09:50:001
382655099139,3cyclictest3919099-21kworker/u8:0+flush-259:009:05:001
382655099139,3cyclictest3847401-21kworker/u8:3+flush-259:008:20:261
382655099139,3cyclictest0-21swapper/110:45:001
3826550991310,2cyclictest498-21dbus-daemon11:22:091
3826550991310,2cyclictest4177702-21cpuspeed_turbos11:35:131
3826550991310,2cyclictest3842010-21sh07:24:591
3826546991310,2cyclictest61950irq/133-enp0s29f1:tx-007:15:350
3826546991310,2cyclictest3919100-21kworker/u8:4+flush-259:008:47:500
382655099129,2cyclictest0-21swapper/111:05:001
382655099128,3cyclictest12141-21ssh11:51:441
382655099126,4cyclictest3878655-21latency_hist08:00:011
382655099121,9cyclictest1-21systemd08:50:451
3826550991210,1cyclictest0-21swapper/111:45:111
382654699129,1cyclictest0-21swapper/007:50:180
382654699128,3cyclictest3904165-21kworker/u8:2+events_unbound08:30:310
382654699128,3cyclictest3847401-21kworker/u8:3+events_unbound08:20:280
382655899119,1cyclictest3925866-21ethtool08:45:193
382655899118,2cyclictest3847401-21kworker/u8:3+events_unbound07:48:013
382655099118,2cyclictest3883851-21cron08:05:001
382655099117,2cyclictest0-21swapper/110:05:191
382655099115,4cyclictest71323-21cut12:30:191
38265509997,1cyclictest0-21swapper/108:40:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional