You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 06:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Mon Jul 27, 2026 00:45:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3360180998717,64cyclictest16850irq/134-800000023:51:050
336018099871,69cyclictest3404273-21ssh22:30:320
336018099871,69cyclictest3404273-21ssh22:30:320
3360180998617,16cyclictest3433977-21ssh23:34:130
3360180998446,34cyclictest16950irq/224-800000000:20:000
3360180998426,41cyclictest3367091-21ssh21:11:120
3360180998420,15cyclictest3448900-21ssh00:05:450
336018599824,62cyclictest3457859-21ssh00:25:021
3360180998228,13cyclictest3456030-21ssh00:20:480
3360180998138,26cyclictest3445691-21ssh23:59:190
336018099811,62cyclictest3446784-21ssh00:01:120
3360180998028,48cyclictest3440667-21ssh23:48:280
3360180998024,51cyclictest3400686-21ssh22:23:120
336018099801,75cyclictest3385775-21ssh21:51:140
336018099801,75cyclictest3385775-21ssh21:51:140
3360180997926,36cyclictest3435467-21ssh23:37:160
3360180997926,36cyclictest3435467-21ssh23:37:150
3360180997819,50cyclictest16850irq/134-800000022:40:040
3360180997814,6cyclictest16850irq/134-800000023:22:180
3360185997721,20cyclictest3383686-21ssh21:46:491
3360180997726,34cyclictest3452661-21ssh00:14:030
336018099771,58cyclictest3430055-21ssh23:25:370
3360180997715,57cyclictest16850irq/134-800000022:02:070
3360180997715,57cyclictest16850irq/134-800000022:02:070
3360185997624,16cyclictest3414250-21ssh22:52:081
3360185997624,16cyclictest3414250-21ssh22:52:081
3360180997627,45cyclictest3437322-21ssh23:41:080
3360180997523,47cyclictest3458951-21ssh00:27:140
3360180997514,42cyclictest16850irq/134-800000021:45:180
3360185997426,16cyclictest3455051-21ssh00:19:041
336018099742,9cyclictest275-21dbus-daemon22:55:010
336018099742,9cyclictest275-21dbus-daemon22:55:010
3360180997429,27cyclictest3419402-21ssh23:03:100
3360180997429,27cyclictest3419402-21ssh23:03:100
3360185997324,16cyclictest3431797-21ssh23:29:351
336018099728,59cyclictest12-21ksoftirqd/021:38:360
3360180997234,34cyclictest16950irq/224-800000023:20:010
3360180997221,2cyclictest0-21swapper/021:42:340
3360180997221,2cyclictest0-21swapper/021:42:340
3360185997124,17cyclictest3419780-21ssh23:03:581
3360185997124,17cyclictest3419780-21ssh23:03:581
3360185997119,18cyclictest3459732-21ssh00:29:021
3360185997119,15cyclictest3462187-21sh00:34:171
3360180997121,45cyclictest0-21swapper/022:08:260
3360185997024,13cyclictest3415840-21sh22:55:211
3360185997019,15cyclictest3463934-21ssh00:37:591
3360185997019,15cyclictest3463934-21ssh00:37:591
3360180997025,41cyclictest0-21swapper/000:38:400
3360180997025,41cyclictest0-21swapper/000:38:390
3360180997017,37cyclictest16850irq/134-800000022:38:070
3360185996927,13cyclictest3403510-21ssh22:29:241
3360185996918,15cyclictest3404761-21ssh22:31:471
3360185996918,15cyclictest3404761-21ssh22:31:461
3360185996918,15cyclictest3382237-21ssh21:43:521
3360185996918,15cyclictest3382237-21ssh21:43:521
3360185996917,16cyclictest3375389-21ssh21:29:141
3360185996916,22cyclictest3365686-21latency_hist20:55:021
3360185996915,15cyclictest3425552-21ssh23:16:131
3360180996915,14cyclictest3374800-21ssh21:27:490
336018099691,12cyclictest3373466-21munin-run21:25:010
3360185996821,14cyclictest3408110-21ssh22:39:141
336018099682,7cyclictest3389756-21munin-run22:00:000
3360180996814,11cyclictest3397736-21ssh22:16:530
3360185996719,14cyclictest3377188-21ssh21:32:531
3360180996727,35cyclictest3424110-21ssh23:13:190
3360180996727,35cyclictest3424110-21ssh23:13:180
3360180996725,38cyclictest0-21swapper/022:29:270
3360185996626,14cyclictest364-21snmpd19:15:301
3360185996618,18cyclictest3369969-21ssh21:17:151
3360185996615,18cyclictest3409699-21ssh22:42:251
3360180996623,39cyclictest0-21swapper/000:33:170
336018099662,16cyclictest3361962-21munin-run19:45:000
3360185996527,11cyclictest3442366-21ssh23:52:071
3360185996518,16cyclictest3397055-21ssh22:15:171
3360185996518,13cyclictest3421607-21rm23:07:551
3360185996516,19cyclictest3434041-21ssh23:34:251
3360185996516,13cyclictest3427643-21ssh23:20:301
3360185996515,16cyclictest3444798-21ssh23:57:161
3360180996530,30cyclictest16950irq/224-800000021:20:000
3360180996520,40cyclictest3376636-21ssh21:31:320
3360185996417,13cyclictest3391350-21ssh22:03:131
3360185996417,13cyclictest3391350-21ssh22:03:131
3360185996415,14cyclictest3411591-21ssh22:46:201
3360185996413,17cyclictest3379413-21ssh21:37:391
336018099642,15cyclictest3363459-21sshd20:11:270
336018099641,59cyclictest0-21swapper/023:09:210
336018099641,59cyclictest0-21swapper/022:56:300
3360185996325,13cyclictest3450628-21ssh00:09:441
3360185996318,17cyclictest3362109-21sed19:45:021
3360185996317,15cyclictest3388027-21ssh21:56:041
3360193996211,35cyclictest3441386-21ssh23:50:013
336018599623,43cyclictest3439160-21ssh23:45:021
3360185996219,14cyclictest3365854-21sh21:00:011
3360185996219,14cyclictest3365854-21sh21:00:011
336018099622,55cyclictest3361639-21sshd19:36:270
336018099622,55cyclictest3361639-21sshd19:36:270
3360180996213,12cyclictest3412142-21ssh22:47:420
3360180996212,14cyclictest3396224-21ssh22:13:450
3360180996212,14cyclictest3396224-21ssh22:13:450
3360185996114,20cyclictest3366355-21diskmemload22:07:571
3360189996049,7cyclictest3385066-21kworker/2:322:38:242
3360185996047,10cyclictest3439261-21ssh23:45:091
3360185996018,13cyclictest3395391-21ssh22:11:461
3360185996018,13cyclictest3395391-21ssh22:11:461
3360185996015,14cyclictest3365298-21sshd20:49:271
3360180996028,28cyclictest3467602-21systemd-journal20:17:000
3360189995949,5cyclictest3446018-21kworker/2:100:40:002
3360189995949,5cyclictest3446018-21kworker/2:100:40:002
3360189995948,6cyclictest3360162-21kworker/2:319:45:002
3360185995948,9cyclictest3452860-21ssh00:14:271
3360185995943,12cyclictest3399924-21ssh22:21:271
3360185995913,13cyclictest3423609-21ssh23:12:051
3360185995913,13cyclictest3423609-21ssh23:12:051
3360185995845,10cyclictest3367910-21ssh21:13:081
3360185995840,14cyclictest3447225-21ssh00:02:231
336018099582,16cyclictest3360336-21sshd19:10:270
3360180995817,9cyclictest12-21ksoftirqd/019:55:020
3360180995817,36cyclictest16850irq/134-800000019:25:400
3360193995742,11cyclictest37-21ksoftirqd/323:00:013
3360185995741,12cyclictest3372341-21ssh21:22:231
3360185995717,14cyclictest3386185-21sh21:52:071
3360185995717,14cyclictest3386185-21sh21:52:071
3360180995712,10cyclictest3361428-21systemd-run19:35:010
336019399567,8cyclictest37-21ksoftirqd/323:40:003
336019399567,8cyclictest37-21ksoftirqd/323:40:003
3360193995611,11cyclictest37-21ksoftirqd/322:26:153
3360189995645,5cyclictest3408533-21kworker/2:223:40:012
3360189995645,5cyclictest3408533-21kworker/2:223:40:012
3360185995638,14cyclictest3434807-21ssh23:35:431
3360185995638,14cyclictest3434807-21ssh23:35:421
336019399551,26cyclictest0-21swapper/321:55:013
336019399551,26cyclictest0-21swapper/321:55:013
3360189995545,6cyclictest3446018-21kworker/2:100:25:072
3360189995544,6cyclictest3385066-21kworker/2:323:18:502
3360185995542,9cyclictest364-21snmpd20:38:261
3360185995542,10cyclictest3364245-21sshd20:25:271
3360185995541,10cyclictest3362434-21sshd19:54:271
3360185995541,10cyclictest3362199-21munin-run19:50:001
3360185995541,10cyclictest3360355-21sshd19:14:271
3360185995539,11cyclictest3361178-21munin-run19:30:011
3360180995522,29cyclictest12-21ksoftirqd/021:01:290
336018099552,14cyclictest3362961-21sshd20:02:270
336018099552,14cyclictest3362961-21sshd20:02:270
3360180995511,5cyclictest0-21swapper/020:55:000
3360193995425,8cyclictest37-21ksoftirqd/300:28:493
3360189995445,5cyclictest3446018-21kworker/2:100:25:012
3360189995444,6cyclictest3408533-21kworker/2:223:46:392
3360185995441,10cyclictest3363714-21cron20:17:001
3360185995436,14cyclictest3363205-21sshd20:06:271
336018099542,14cyclictest3360892-21sshd19:23:270
336018099542,14cyclictest3360892-21sshd19:23:270
336019399533,10cyclictest37-21ksoftirqd/322:20:003
3360189995345,6cyclictest3385066-21kworker/2:322:44:562
3360189995344,5cyclictest3371107-21kworker/2:022:16:302
3360189995344,5cyclictest3364838-21kworker/2:221:25:312
3360189995334,15cyclictest1-21systemd19:50:012
3360185995340,10cyclictest3366315-21sshd21:05:271
3360185995340,10cyclictest3361430-21munin-run19:35:011
3360180995332,15cyclictest275-21dbus-daemon20:45:010
3360180995332,15cyclictest275-21dbus-daemon20:45:010
336018099532,17cyclictest3365845-21sendmail-msp21:00:000
336018099532,17cyclictest3365845-21sendmail-msp21:00:000
3360180995314,7cyclictest12-21ksoftirqd/019:50:020
336019399525,7cyclictest37-21ksoftirqd/300:33:453
336019399521,20cyclictest0-21swapper/322:04:353
336019399521,20cyclictest0-21swapper/322:04:353
3360189995244,5cyclictest3364838-21kworker/2:221:34:432
3360189995240,7cyclictest3385066-21kworker/2:323:58:522
3360185995239,9cyclictest48-21kcompactd020:03:471
3360185995239,9cyclictest48-21kcompactd020:03:471
3360185995237,11cyclictest3365065-21munin-run20:45:011
3360185995237,11cyclictest3365065-21munin-run20:45:011
3360185995237,10cyclictest48-21kcompactd019:22:441
3360185995237,10cyclictest48-21kcompactd019:22:441
3360180995235,13cyclictest275-21dbus-daemon20:50:000
336018099521,46cyclictest0-21swapper/021:10:010
336019399518,6cyclictest131rcu_preempt21:18:163
336019399518,6cyclictest131rcu_preempt00:18:373
3360193995118,15cyclictest37-21ksoftirqd/323:56:053
3360193995111,6cyclictest131rcu_preempt21:32:273
336019399511,15cyclictest3406159-21true22:35:013
336019399511,15cyclictest3406159-21true22:35:013
3360189995141,7cyclictest3364838-21kworker/2:221:48:152
3360185995134,12cyclictest3362682-21sshd19:58:271
3360185995130,17cyclictest3363467-21sshd20:13:271
336018099519,10cyclictest1-21systemd19:20:010
336018099515,11cyclictest3467602-21systemd-journal20:00:000
3360193995038,10cyclictest37-21ksoftirqd/322:42:583
3360193995038,10cyclictest37-21ksoftirqd/322:22:463
3360193995033,13cyclictest3467718-21lldpd20:25:003
336019399501,7cyclictest3415460-21munin-plugin-st22:55:013
336019399501,7cyclictest3415460-21munin-plugin-st22:55:003
3360189995043,4cyclictest3385066-21kworker/2:322:48:402
3360189995042,5cyclictest3385066-21kworker/2:300:05:422
3360189995042,5cyclictest3371107-21kworker/2:022:28:452
3360189995041,5cyclictest3371107-21kworker/2:021:56:502
3360189995040,6cyclictest3364838-21kworker/2:221:44:312
3360189995040,6cyclictest3364838-21kworker/2:221:44:312
3360189995040,6cyclictest3362757-21kworker/2:120:07:212
3360185995037,9cyclictest48-21kcompactd019:38:161
3360185995037,9cyclictest48-21kcompactd019:38:161
3360185995034,11cyclictest0-21swapper/120:25:001
336019399496,7cyclictest37-21ksoftirqd/321:28:433
336019399496,11cyclictest37-21ksoftirqd/321:41:533
336019399496,11cyclictest37-21ksoftirqd/321:41:533
336019399495,8cyclictest37-21ksoftirqd/323:51:293
3360193994928,15cyclictest275-21dbus-daemon23:25:013
3360193994910,6cyclictest131rcu_preempt22:45:323
3360193994910,6cyclictest131rcu_preempt19:50:003
3360189994941,5cyclictest3446018-21kworker/2:100:34:432
3360189994941,5cyclictest3446018-21kworker/2:100:17:512
3360189994941,5cyclictest3385066-21kworker/2:323:50:512
3360189994941,5cyclictest3385066-21kworker/2:323:21:582
3360189994941,5cyclictest3364838-21kworker/2:221:24:492
3360189994940,6cyclictest3385066-21kworker/2:322:50:222
3360189994940,6cyclictest3385066-21kworker/2:322:50:222
3360189994940,5cyclictest3385066-21kworker/2:323:00:512
3360189994940,5cyclictest3385066-21kworker/2:323:00:512
3360189994929,15cyclictest275-21dbus-daemon22:15:012
3360189994929,15cyclictest275-21dbus-daemon22:15:012
3360189994927,15cyclictest275-21dbus-daemon21:10:012
336018099493,9cyclictest3364303-21grep20:30:010
336019399487,10cyclictest37-21ksoftirqd/321:13:533
3360193994813,11cyclictest37-21ksoftirqd/322:11:033
3360193994813,11cyclictest37-21ksoftirqd/322:11:033
3360193994810,5cyclictest131rcu_preempt00:40:003
3360193994810,5cyclictest131rcu_preempt00:40:003
3360189994841,4cyclictest3364838-21kworker/2:221:38:142
3360189994838,6cyclictest3362757-21kworker/2:120:37:272
3360189994837,7cyclictest3408533-21kworker/2:223:45:002
3360185994832,12cyclictest364-21snmpd20:34:521
336018099481,6cyclictest3467602-21systemd-journal20:30:320
336019399476,8cyclictest37-21ksoftirqd/300:09:293
336019399476,19cyclictest37-21ksoftirqd/322:05:033
336019399475,10cyclictest37-21ksoftirqd/323:18:053
3360193994736,9cyclictest37-21ksoftirqd/300:04:283
3360193994736,7cyclictest37-21ksoftirqd/323:30:003
3360193994730,14cyclictest3421289-21rm23:07:053
3360189994739,5cyclictest3385066-21kworker/2:323:08:122
3360189994739,5cyclictest3362757-21kworker/2:121:19:512
3360189994738,5cyclictest3361191-21kworker/2:019:51:272
3360189994729,13cyclictest275-21dbus-daemon20:05:012
3360189994729,13cyclictest275-21dbus-daemon20:05:012
3360189994728,14cyclictest3425011-21wc23:15:022
3360189994728,14cyclictest3425011-21wc23:15:022
3360185994732,12cyclictest364-21snmpd21:00:551
3360193994636,7cyclictest361rcuc/322:36:413
3360193994632,11cyclictest1-21systemd20:40:003
336019399463,11cyclictest37-21ksoftirqd/321:56:333
336019399461,9cyclictest37-21ksoftirqd/300:12:323
3360193994614,9cyclictest131rcu_preempt23:12:333
3360193994614,9cyclictest131rcu_preempt23:12:333
3360193994611,6cyclictest131rcu_preempt23:43:243
3360193994611,5cyclictest131rcu_preempt21:45:163
336018999468,6cyclictest131rcu_preempt00:03:112
3360189994638,5cyclictest3385066-21kworker/2:323:27:202
3360189994638,5cyclictest3385066-21kworker/2:322:32:492
3360189994638,5cyclictest3385066-21kworker/2:322:32:482
3360189994638,5cyclictest3371107-21kworker/2:022:04:432
3360189994638,5cyclictest3371107-21kworker/2:022:04:432
3360189994638,5cyclictest3362757-21kworker/2:121:10:062
3360189994638,5cyclictest3360162-21kworker/2:319:35:002
3360189994637,5cyclictest3362757-21kworker/2:120:45:272
3360189994637,5cyclictest3362757-21kworker/2:120:45:012
3360189994637,5cyclictest3362757-21kworker/2:120:45:012
3360189994636,5cyclictest3362757-21kworker/2:121:00:022
3360189994636,5cyclictest3362757-21kworker/2:121:00:022
3360189994631,11cyclictest275-21dbus-daemon20:15:012
3360180994620,22cyclictest12-21ksoftirqd/020:40:010
336019399456,9cyclictest37-21ksoftirqd/323:04:393
336019399456,9cyclictest37-21ksoftirqd/323:04:383
336019399456,5cyclictest131rcu_preempt21:10:003
3360193994536,7cyclictest37-21ksoftirqd/323:33:133
3360193994512,10cyclictest37-21ksoftirqd/300:22:433
3360189994540,3cyclictest3385066-21kworker/2:323:33:532
3360189994539,3cyclictest3371107-21kworker/2:022:07:232
3360189994537,5cyclictest3371107-21kworker/2:021:52:562
3360189994537,5cyclictest3371107-21kworker/2:021:52:552
3360189994537,5cyclictest3362757-21kworker/2:121:03:302
3360189994524,16cyclictest275-21dbus-daemon20:35:012
336018099451,4cyclictest0-21swapper/020:21:190
336019399445,5cyclictest131rcu_preempt21:20:443
336019399444,5cyclictest131rcu_preempt20:06:273
336019399441,7cyclictest150-21pr/ttyS221:04:073
3360189994436,5cyclictest3385066-21kworker/2:322:56:272
336018099441,23cyclictest0-21swapper/020:06:490
3360193994329,7cyclictest3360911-21munin-plugin-st19:25:003
3360193994329,7cyclictest3360911-21munin-plugin-st19:24:593
336019399432,10cyclictest37-21ksoftirqd/321:38:113
336019399432,10cyclictest37-21ksoftirqd/320:00:023
3360189994332,6cyclictest3446018-21kworker/2:100:10:492
3360193994232,5cyclictest309-21in:imuxsock20:45:013
3360193994232,5cyclictest309-21in:imuxsock20:45:013
3360193994229,10cyclictest37-21ksoftirqd/319:30:003
336019399420,7cyclictest150-21pr/ttyS220:27:203
336018999426,5cyclictest131rcu_preempt20:00:002
3360189994234,5cyclictest3371107-21kworker/2:022:20:052
3360189994234,5cyclictest3364838-21kworker/2:220:51:252
3360189994234,5cyclictest3360162-21kworker/2:319:37:062
3360189994234,5cyclictest3360162-21kworker/2:319:37:062
3360193994131,7cyclictest3361913-21sshd19:40:273
3360193994127,11cyclictest1-21systemd20:55:003
336019399411,7cyclictest0-21swapper/319:40:023
336019399411,7cyclictest0-21swapper/319:40:023
336019399411,5cyclictest0-21swapper/320:34:273
3360189994133,5cyclictest3352094-21kworker/2:119:10:572
3360189994131,6cyclictest3362757-21kworker/2:120:16:282
3360185994123,14cyclictest3360115-21latency_hist19:10:011
3360193994031,7cyclictest37-21ksoftirqd/320:20:013
336019399400,6cyclictest150-21pr/ttyS219:18:153
336019399398,6cyclictest131rcu_preempt19:35:013
3360193993925,10cyclictest275-21dbus-daemon21:00:013
3360193993925,10cyclictest275-21dbus-daemon21:00:013
3360193993911,5cyclictest131rcu_preempt19:55:023
3360189993929,8cyclictest30-21ksoftirqd/219:25:002
3360189993929,8cyclictest30-21ksoftirqd/219:25:002
336019399381,7cyclictest150-21pr/ttyS220:01:173
336019399381,7cyclictest150-21pr/ttyS220:01:173
336019399381,6cyclictest150-21pr/ttyS220:49:213
336019399381,5cyclictest0-21swapper/319:10:023
3360189993830,5cyclictest3362757-21kworker/2:120:27:442
3360189993828,8cyclictest30-21ksoftirqd/220:25:022
3360189993826,9cyclictest3467602-21systemd-journal19:29:022
3360189993825,9cyclictest3467602-21systemd-journal19:20:002
3360193993726,8cyclictest361rcuc/320:10:273
336019399361,11cyclictest37-21ksoftirqd/319:10:013
3360180993422,9cyclictest0-21swapper/019:10:010
3360189993222,6cyclictest3360258-21latency_hist19:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional