You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 16:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Jul 15, 2026 12:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
379131299948,11cyclictest131rcu_preempt10:20:043
379131299948,11cyclictest131rcu_preempt10:20:033
3791297997166,3cyclictest0-21swapper/010:00:030
3791312996924,41cyclictest1-21systemd12:30:013
3791312996631,30cyclictest275-21dbus-daemon09:05:013
3791297996641,9cyclictest12-21ksoftirqd/011:42:210
379129799662,60cyclictest0-21swapper/008:55:030
379131299657,52cyclictest1-21systemd09:20:023
3791312996524,38cyclictest3859327-21ssh11:19:573
3791297996518,10cyclictest12-21ksoftirqd/012:18:000
3791297996515,11cyclictest12-21ksoftirqd/008:10:010
3791312996427,33cyclictest3878817-21sh12:00:123
3791312996319,12cyclictest275-21dbus-daemon09:35:003
3791297996337,17cyclictest275-21dbus-daemon09:50:010
379131299622,55cyclictest3878476-21cron12:00:013
3791312996218,5cyclictest3897741-21latency_hist12:40:023
3791307996228,8cyclictest3799806-21kworker/2:010:46:242
3791297996235,23cyclictest16950irq/224-800000009:20:020
379131299602,21cyclictest3857025-21cron11:15:003
3791302996023,31cyclictest23-21ksoftirqd/112:40:021
3791297996027,30cyclictest3873427-21ssh11:49:200
379131299593,17cyclictest3854512-21ssh11:09:423
3791312995923,8cyclictest37-21ksoftirqd/310:48:473
3791312995917,8cyclictest37-21ksoftirqd/312:09:093
3791312995916,7cyclictest37-21ksoftirqd/311:30:333
379130299594,45cyclictest3804782-21ssh09:25:021
3791297995923,31cyclictest16850irq/134-800000009:30:250
3791297995916,8cyclictest12-21ksoftirqd/010:58:100
3791297995915,7cyclictest3859734-21ssh11:20:170
379129799591,28cyclictest0-21swapper/012:29:090
379131299587,17cyclictest3837910-21modprobe10:35:013
379131299587,17cyclictest3837910-21modprobe10:35:003
3791312995824,30cyclictest3847441-21sh10:54:583
379131299582,21cyclictest3794870-21cron08:20:003
3791312995821,8cyclictest37-21ksoftirqd/311:05:013
3791312995818,8cyclictest37-21ksoftirqd/309:12:593
3791307995850,5cyclictest3784084-21kworker/2:109:48:352
3791297995848,7cyclictest16950irq/224-800000010:05:310
3791297995829,14cyclictest3894821-21ssh12:34:170
379129799582,15cyclictest0-21swapper/010:39:280
3791297995811,9cyclictest12-21ksoftirqd/010:24:060
3791297995811,9cyclictest12-21ksoftirqd/010:24:050
3791312995726,28cyclictest37-21ksoftirqd/311:50:443
3791312995722,7cyclictest37-21ksoftirqd/310:38:403
3791312995716,8cyclictest3794670-21date08:15:013
3791312995716,8cyclictest3794670-21date08:15:013
3791307995749,5cyclictest3862095-21kworker/2:211:41:042
3791297995729,12cyclictest3833568-21ssh10:25:270
3791297995728,13cyclictest3840989-21ssh10:41:050
3791297995727,26cyclictest16850irq/134-800000009:26:520
3791297995723,30cyclictest16850irq/134-800000010:15:430
3791297995713,7cyclictest12-21ksoftirqd/009:44:560
3791297995713,7cyclictest12-21ksoftirqd/009:44:550
3791312995624,28cyclictest3835117-21sh10:29:013
379131299562,18cyclictest3791745-21cron07:19:593
3791312995620,4cyclictest3849435-21sh10:59:033
3791312995617,11cyclictest37-21ksoftirqd/312:10:463
3791312995616,5cyclictest3841934-21rm10:43:113
3791312995615,7cyclictest3793368-21sed07:50:023
3791307995636,12cyclictest3893717-21rm12:31:392
3791297995628,13cyclictest3881217-21ssh12:05:220
3791297995627,12cyclictest3857312-21ssh11:15:080
379129799562,38cyclictest3856110-21ssh11:12:580
3791297995617,7cyclictest12-21ksoftirqd/012:37:590
3791297995617,5cyclictest3798347-21ssh09:11:410
3791297995615,37cyclictest3868211-21ssh11:38:180
379131299554,11cyclictest3811761-21cron09:39:593
3791312995532,18cyclictest275-21dbus-daemon08:55:003
379131299553,18cyclictest3467602-21systemd-journal09:53:393
3791312995526,25cyclictest3795227-21latency_hist08:25:013
3791312995526,25cyclictest3795227-21latency_hist08:25:013
3791307995548,4cyclictest3799806-21kworker/2:010:53:092
3791307995547,5cyclictest3862095-21kworker/2:212:19:322
3791307995547,5cyclictest3784084-21kworker/2:109:10:012
3791307995546,6cyclictest3799806-21kworker/2:009:55:102
3791307995546,6cyclictest3784084-21kworker/2:109:11:002
379129799556,41cyclictest3793832-21modprobe07:59:590
3791297995514,9cyclictest12-21ksoftirqd/012:13:390
3791297995511,40cyclictest3845221-21latency_hist10:50:020
3791312995427,24cyclictest3889380-21sh12:22:433
379131299542,18cyclictest3812747-21ssh09:41:383
379131299542,18cyclictest3812747-21ssh09:41:373
3791312995415,12cyclictest3888111-21sort12:20:023
3791307995446,5cyclictest3862095-21kworker/2:212:02:512
3791307995446,5cyclictest3862095-21kworker/2:211:53:362
3791307995445,5cyclictest3862095-21kworker/2:212:26:382
3791307995442,7cyclictest3799806-21kworker/2:010:45:022
3791302995436,14cyclictest3467602-21systemd-journal09:30:001
379129799545,15cyclictest3837009-21ssh10:32:590
379129799545,15cyclictest3837009-21ssh10:32:590
3791297995445,5cyclictest16950irq/224-800000012:05:010
379129799544,46cyclictest0-21swapper/012:23:010
3791297995431,20cyclictest3810060-21ssh09:36:130
3791297995411,14cyclictest3863827-21ssh11:29:110
3791312995324,26cyclictest37-21ksoftirqd/312:31:153
3791312995323,9cyclictest37-21ksoftirqd/310:07:053
3791312995317,7cyclictest37-21ksoftirqd/311:30:023
379131299531,21cyclictest3817141-21kworker/u8:010:05:023
3791307995345,5cyclictest3862095-21kworker/2:212:22:462
3791307995345,5cyclictest3862095-21kworker/2:212:07:182
3791307995344,6cyclictest3864219-21kworker/2:312:38:352
3791307995344,6cyclictest3799806-21kworker/2:010:38:142
379129799536,13cyclictest3796699-21latency_hist08:55:010
3791297995342,6cyclictest16950irq/224-800000007:40:000
3791297995342,6cyclictest16950irq/224-800000007:40:000
3791297995326,12cyclictest3821252-21ssh09:59:470
3791297995325,12cyclictest3853356-21ssh11:07:000
3791297995316,5cyclictest131rcu_preempt11:35:010
3791307995244,5cyclictest3862095-21kworker/2:211:55:312
3791307995244,5cyclictest3799806-21kworker/2:011:05:382
3791307995244,5cyclictest3784084-21kworker/2:109:16:212
3791307995243,6cyclictest3862095-21kworker/2:211:30:322
3791307995243,6cyclictest3799806-21kworker/2:011:23:462
379129799522,34cyclictest3827563-21ssh10:13:100
3791297995214,8cyclictest12-21ksoftirqd/008:15:030
3791297995214,8cyclictest12-21ksoftirqd/008:15:020
379131299513,19cyclictest3796550-21grep08:50:023
3791312995122,7cyclictest37-21ksoftirqd/307:58:523
3791312995121,9cyclictest37-21ksoftirqd/309:20:283
3791312995121,8cyclictest37-21ksoftirqd/307:34:353
3791312995120,7cyclictest37-21ksoftirqd/309:29:503
3791312995120,7cyclictest37-21ksoftirqd/307:50:083
3791312995120,7cyclictest37-21ksoftirqd/307:50:073
3791312995119,9cyclictest37-21ksoftirqd/311:39:023
3791312995114,9cyclictest3795395-21sshd08:28:423
3791307995143,5cyclictest3862095-21kworker/2:212:10:592
3791307995143,5cyclictest3862095-21kworker/2:211:45:272
3791307995143,5cyclictest3799806-21kworker/2:011:26:532
3791307995143,5cyclictest3799806-21kworker/2:011:12:482
3791307995143,5cyclictest3799806-21kworker/2:010:15:302
3791307995143,5cyclictest3799806-21kworker/2:009:55:002
3791307995143,5cyclictest3799806-21kworker/2:009:41:022
3791307995143,5cyclictest3799806-21kworker/2:009:41:012
3791307995143,5cyclictest3799806-21kworker/2:009:38:032
3791307995143,5cyclictest3799806-21kworker/2:009:23:032
3791307995142,6cyclictest3799806-21kworker/2:011:20:002
379129799518,10cyclictest3804038-21ssh09:23:460
3791297995114,9cyclictest1-21systemd08:36:390
3791297995114,9cyclictest1-21systemd08:36:380
379131299503,13cyclictest3796949-21modprobe08:59:593
3791312995018,9cyclictest37-21ksoftirqd/310:14:003
3791312995018,8cyclictest37-21ksoftirqd/308:43:453
3791312995018,4cyclictest3795921-21latency_hist08:40:033
3791312995018,4cyclictest3795921-21latency_hist08:40:023
379131299501,10cyclictest37-21ksoftirqd/309:56:053
3791312995010,13cyclictest275-21dbus-daemon11:45:013
3791307995042,5cyclictest3799806-21kworker/2:010:55:522
3791307995042,5cyclictest3799806-21kworker/2:010:30:372
3791307995042,5cyclictest3799806-21kworker/2:010:30:372
3791307995042,5cyclictest3799806-21kworker/2:010:03:302
3791307995042,5cyclictest3799806-21kworker/2:009:30:482
3791307995041,6cyclictest3862095-21kworker/2:211:36:472
3791307995041,6cyclictest3799806-21kworker/2:010:23:582
3791307995041,6cyclictest3799806-21kworker/2:010:23:582
3791307995041,6cyclictest3784084-21kworker/2:108:55:002
3791307995041,5cyclictest3799806-21kworker/2:010:09:092
3791307995041,5cyclictest3784084-21kworker/2:107:35:012
3791307995040,6cyclictest3799806-21kworker/2:010:11:102
3791297995027,19cyclictest3851904-21ssh11:04:140
379129799501,34cyclictest3874168-21ssh11:50:300
3791312994913,11cyclictest37-21ksoftirqd/310:16:083
3791307994941,5cyclictest3799806-21kworker/2:010:29:552
3791307994941,5cyclictest3799806-21kworker/2:009:27:542
3791307994941,5cyclictest3784084-21kworker/2:111:02:362
3791307994930,14cyclictest3795935-21tr08:40:002
3791307994930,14cyclictest3795935-21tr08:40:002
379129799492,17cyclictest3791764-21munin-run07:20:000
3791312994822,4cyclictest3861392-21sh11:24:083
3791312994818,7cyclictest37-21ksoftirqd/308:10:033
3791312994815,8cyclictest37-21ksoftirqd/309:47:053
3791307994841,5cyclictest3784084-21kworker/2:107:30:022
3791307994841,5cyclictest3784084-21kworker/2:107:30:012
3791307994840,5cyclictest3784084-21kworker/2:107:46:392
3791307994739,5cyclictest3784084-21kworker/2:108:45:452
3791307994739,5cyclictest3784084-21kworker/2:107:56:132
3791307994738,6cyclictest3784084-21kworker/2:108:07:052
3791302994732,10cyclictest0-21swapper/111:05:011
3791302994729,13cyclictest3467602-21systemd-journal11:30:011
379129799472,14cyclictest3818461-21sshd09:53:550
3791297994710,33cyclictest3795146-21sshd08:23:410
3791297994710,33cyclictest3795146-21sshd08:23:400
3791307994639,4cyclictest3784084-21kworker/2:109:03:032
3791307994639,4cyclictest3784084-21kworker/2:108:30:552
3791307994639,4cyclictest3784084-21kworker/2:107:43:222
3791307994638,5cyclictest3784084-21kworker/2:108:18:112
3791307994638,5cyclictest3784084-21kworker/2:107:13:522
3791302994638,6cyclictest23-21ksoftirqd/109:35:031
3791302994630,12cyclictest275-21dbus-daemon09:10:001
3791302994629,12cyclictest275-21dbus-daemon11:55:001
379129799464,37cyclictest3796194-21latency_hist08:45:020
3791312994521,6cyclictest37-21ksoftirqd/311:47:283
3791307994538,4cyclictest3784084-21kworker/2:107:16:592
3791307994537,5cyclictest3784084-21kworker/2:108:56:212
3791307994537,5cyclictest3784084-21kworker/2:107:22:302
379130299458,19cyclictest3792767-21kworker/u8:207:55:021
379130299458,19cyclictest3792767-21kworker/u8:207:55:021
3791302994531,11cyclictest3467602-21systemd-journal12:05:411
3791302994521,5cyclictest23-21ksoftirqd/109:12:401
3791297994530,11cyclictest275-21dbus-daemon09:10:010
379129799451,15cyclictest0-21swapper/010:54:290
3791307994436,5cyclictest3784084-21kworker/2:108:29:502
3791307994436,5cyclictest3784084-21kworker/2:108:10:232
3791307994436,5cyclictest3784084-21kworker/2:108:10:222
3791307994436,5cyclictest3784084-21kworker/2:108:02:432
379130299447,5cyclictest131rcu_preempt10:25:011
379130299447,5cyclictest131rcu_preempt10:25:001
3791302994429,10cyclictest275-21dbus-daemon07:25:001
3791302994427,12cyclictest275-21dbus-daemon11:25:011
379130299442,3cyclictest131rcu_preempt12:25:021
379130299441,3cyclictest131rcu_preempt10:09:101
3791297994428,13cyclictest1-21systemd07:25:010
3791297994426,15cyclictest3877437-21ssh11:57:400
3791307994335,5cyclictest3784084-21kworker/2:108:23:402
3791307994335,5cyclictest3784084-21kworker/2:108:23:392
3791307994335,5cyclictest3784084-21kworker/2:107:50:382
3791307994335,5cyclictest3784084-21kworker/2:107:50:372
3791307994327,12cyclictest3792804-21run-parts07:40:002
3791307994327,12cyclictest3792804-21run-parts07:40:002
3791302994330,9cyclictest1-21systemd12:15:011
3791302994327,12cyclictest3467602-21systemd-journal10:30:031
3791302994327,12cyclictest3467602-21systemd-journal10:30:021
3791302994316,7cyclictest3830762-21kworker/1:111:10:021
3791297994330,10cyclictest3794842-21sshd08:16:400
3791297994330,10cyclictest3793568-21sshd07:54:400
3791297994330,10cyclictest3793568-21sshd07:54:390
3791307994234,5cyclictest3784084-21kworker/2:108:40:212
3791302994230,8cyclictest23-21ksoftirqd/108:30:001
3791302994230,6cyclictest309-21in:imuxsock10:50:031
3791302994225,12cyclictest275-21dbus-daemon08:25:021
3791302994225,12cyclictest275-21dbus-daemon08:25:011
3791302994220,18cyclictest23-21ksoftirqd/110:49:451
379130299421,3cyclictest131rcu_preempt09:17:291
3791297994227,11cyclictest3791463-21sshd07:10:390
3791297994225,12cyclictest275-21dbus-daemon08:35:000
3791297994210,27cyclictest3792317-21kworker/0:207:30:020
3791297994210,27cyclictest3792317-21kworker/0:207:30:020
3791302994128,8cyclictest3467602-21systemd-journal10:59:591
3791302994126,11cyclictest23-21ksoftirqd/109:35:281
3791297994128,10cyclictest3792546-21munin-run07:35:010
379129799411,4cyclictest0-21swapper/007:48:380
3791312994028,8cyclictest3792026-21cron07:25:003
379130299403,3cyclictest131rcu_preempt12:02:181
3791302994030,5cyclictest3791517-21latency_hist07:15:011
3791302994027,10cyclictest3828203-21rm10:14:371
3791297994032,5cyclictest3791309-21ls07:10:020
3791312993928,8cyclictest37-21ksoftirqd/307:15:033
3791312993926,10cyclictest1-21systemd07:30:013
3791312993926,10cyclictest1-21systemd07:30:003
3791302993928,9cyclictest23-21ksoftirqd/110:42:411
3791302993927,8cyclictest23-21ksoftirqd/111:45:021
3791302993926,10cyclictest23-21ksoftirqd/110:03:541
3791302993925,11cyclictest23-21ksoftirqd/110:18:311
3791302993924,11cyclictest3867615-21ssh11:37:001
379129799395,7cyclictest12-21ksoftirqd/008:03:390
3791297993931,3cyclictest0-21swapper/009:05:010
3791297993921,14cyclictest3795390-21sshd08:28:390
379129799391,34cyclictest0-21swapper/007:45:010
3791312993822,12cyclictest3794098-21sshd08:03:393
3791302993826,9cyclictest23-21ksoftirqd/111:11:111
3791302993825,9cyclictest3794404-21date08:10:011
3791302993822,12cyclictest3797490-21diskmemload11:17:551
3791302993820,14cyclictest275-21dbus-daemon11:35:001
3791302993819,14cyclictest275-21dbus-daemon12:20:001
3791302993810,16cyclictest23-21ksoftirqd/109:44:461
3791302993810,16cyclictest23-21ksoftirqd/109:44:461
3791302993810,15cyclictest23-21ksoftirqd/112:29:551
3791297993824,11cyclictest3796423-21sshd08:47:380
3791312993723,10cyclictest37-21ksoftirqd/309:10:013
379130299375,29cyclictest3877429-21sshd11:57:411
379130299374,3cyclictest131rcu_preempt09:48:561
3791302993730,4cyclictest3796414-21sshd08:45:381
3791302993725,10cyclictest23-21ksoftirqd/110:26:431
3791302993723,11cyclictest23-21ksoftirqd/110:36:541
3791302993722,10cyclictest3467602-21systemd-journal09:05:011
3791302993721,12cyclictest309-21in:imuxsock08:18:411
3791302993718,4cyclictest23-21ksoftirqd/109:54:111
3791302993713,18cyclictest275-21dbus-daemon07:30:021
3791302993713,18cyclictest275-21dbus-daemon07:30:021
379131299361,3cyclictest131rcu_preempt08:35:003
3791302993626,8cyclictest23-21ksoftirqd/111:46:461
3791302993626,7cyclictest23-21ksoftirqd/107:10:011
3791302993623,8cyclictest0-21swapper/107:20:011
3791302993617,14cyclictest275-21dbus-daemon07:40:011
3791302993617,14cyclictest275-21dbus-daemon07:40:011
3791302993614,16cyclictest3467602-21systemd-journal08:45:001
3791302993611,21cyclictest3796983-21modprobe08:59:591
3791302993526,4cyclictest3793840-21sendmail08:00:001
379130299352,5cyclictest131rcu_preempt12:30:441
3791302993523,8cyclictest23-21ksoftirqd/108:39:141
3791302993523,8cyclictest23-21ksoftirqd/108:39:131
3791302993522,9cyclictest23-21ksoftirqd/108:54:341
3791302993521,10cyclictest23-21ksoftirqd/107:44:591
3791302993520,12cyclictest3820236-21sh09:57:221
3791302993513,18cyclictest23-21ksoftirqd/107:35:011
3791312993423,7cyclictest37-21ksoftirqd/307:43:173
3791302993422,9cyclictest23-21ksoftirqd/108:13:341
3791302993422,9cyclictest23-21ksoftirqd/108:13:331
3791312993325,4cyclictest3791241-21latency_hist07:10:023
3791302993222,7cyclictest23-21ksoftirqd/107:47:131
3791302993221,8cyclictest3794254-21wc08:05:021
3791312993122,6cyclictest37-21ksoftirqd/307:35:033
3791312993122,6cyclictest37-21ksoftirqd/307:35:023
3791302993119,8cyclictest3795848-21sort08:35:021
3791307992917,8cyclictest271irq_work/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional