You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-06 - 06:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot2s.osadl.org (updated Sat Jun 06, 2026 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3595869997870,5cyclictest0-21swapper/220:56:302
3595869997870,5cyclictest0-21swapper/220:56:302
3595867997466,5cyclictest86-21hwrng23:04:300
3595869997165,3cyclictest0-21swapper/221:57:572
3595870997062,5cyclictest86-21hwrng21:28:153
3595869997065,3cyclictest0-21swapper/221:39:312
3595869997063,4cyclictest0-21swapper/221:45:402
3595869997063,4cyclictest0-21swapper/221:45:402
3595870996825,38cyclictest3714263-21ssh23:15:113
3595867996760,4cyclictest86-21hwrng22:10:140
3595869996660,3cyclictest0-21swapper/222:11:162
3595870996559,4cyclictest86-21hwrng23:38:183
3595870996559,4cyclictest86-21hwrng23:38:183
359587099653,19cyclictest3694213-21ssh22:43:423
359587099652,8cyclictest0-21swapper/323:45:493
359587099642,38cyclictest3718171-21/usr/sbin/munin23:20:593
359587099642,38cyclictest3718171-21/usr/sbin/munin23:20:593
3595870996421,39cyclictest0-21swapper/319:15:503
3595870996320,40cyclictest0-21swapper/322:51:173
3595870996320,40cyclictest0-21swapper/322:51:173
3595870996320,23cyclictest3737157-21/usr/sbin/munin23:50:293
3595870996319,8cyclictest0-21swapper/323:10:533
3595870996319,8cyclictest0-21swapper/323:10:533
3595870996319,10cyclictest0-21swapper/300:15:513
359587099631,21cyclictest3765738-21ssh00:35:233
359587099631,13cyclictest3676884-21/usr/sbin/munin22:16:003
359587099631,13cyclictest3676884-21/usr/sbin/munin22:16:003
359587099622,20cyclictest3673420-21ssh22:10:473
3595870996219,9cyclictest0-21swapper/322:55:083
3595870996216,11cyclictest0-21swapper/320:10:503
359587099621,14cyclictest3747018-21irqstats00:05:473
359587099612,19cyclictest3652873-21apt-get21:40:203
359587099612,19cyclictest3652873-21apt-get21:40:203
3595870996121,37cyclictest0-21swapper/323:05:493
3595870996121,36cyclictest0-21swapper/300:20:453
3595870996119,38cyclictest0-21swapper/323:55:303
3595870996119,23cyclictest3634635-21ssh21:10:363
3595867996154,4cyclictest86-21hwrng22:15:210
3595867996154,4cyclictest86-21hwrng22:15:210
3595870996019,9cyclictest0-21swapper/300:10:583
3595870996017,9cyclictest0-21swapper/319:35:543
3595870996016,27cyclictest3686315-21ssh22:30:493
3595870996016,10cyclictest0-21swapper/320:00:323
359587099598,48cyclictest0-21swapper/319:50:263
359587099592,21cyclictest3759233-21/usr/sbin/munin00:25:223
3595870995920,34cyclictest0-21swapper/322:20:373
3595870995915,13cyclictest0-21swapper/319:30:073
3595868995952,4cyclictest0-21swapper/120:56:301
3595868995952,4cyclictest0-21swapper/120:56:301
3595870995850,5cyclictest86-21hwrng23:24:593
3595870995821,8cyclictest0-21swapper/319:10:443
3595870995820,35cyclictest0-21swapper/322:01:253
3595870995820,35cyclictest0-21swapper/322:01:253
3595870995819,34cyclictest0-21swapper/321:54:583
359587099581,24cyclictest3731320-21ssh23:40:553
3595869995852,3cyclictest0-21swapper/222:08:112
3595867995848,7cyclictest86-21hwrng21:56:550
3595870995749,5cyclictest86-21hwrng19:25:223
359587099572,22cyclictest3660558-21ntpq21:50:503
3595870995721,9cyclictest0-21swapper/322:25:203
3595870995718,34cyclictest0-21swapper/322:05:413
3595870995717,10cyclictest0-21swapper/321:15:443
3595870995715,8cyclictest0-21swapper/323:00:473
359587099571,52cyclictest3657067-21ssh21:45:363
359587099571,52cyclictest3657067-21ssh21:45:363
3595870995714,9cyclictest0-21swapper/300:30:503
3595870995714,25cyclictest3647316-21iwlist21:30:313
359587099571,38cyclictest3744036-21ssh00:00:483
359587099571,38cyclictest3744036-21ssh00:00:483
359587099571,38cyclictest3619872-21/usr/sbin/munin20:25:223
3595870995712,11cyclictest0-21swapper/320:15:533
359587099569,4cyclictest0-21swapper/320:20:073
359587099566,20cyclictest3695877-21ssh22:45:433
359587099566,20cyclictest3695877-21ssh22:45:433
359587099563,23cyclictest3723899-21ssh23:30:113
359587099562,24cyclictest3641367-21ntpq21:20:583
3595870995620,21cyclictest3650427-21ssh21:35:563
359587099561,20cyclictest3688497-21/usr/sbin/munin22:35:073
359587099552,23cyclictest3630576-21munin-node21:00:063
359587099552,22cyclictest3626141-21modprobe20:45:253
3595870995516,20cyclictest3622524-21sshd-session20:34:253
3595870995516,20cyclictest3622524-21sshd-session20:34:253
3595870995514,9cyclictest0-21swapper/319:45:413
359587099551,24cyclictest3625052-21ntpq20:40:383
359587099551,22cyclictest3599849-21/usr/sbin/munin19:20:333
359587099551,22cyclictest3599849-21/usr/sbin/munin19:20:333
3595870995418,17cyclictest3627447-21/usr/sbin/munin20:50:073
3595870995418,17cyclictest3627447-21/usr/sbin/munin20:50:073
3595870995414,9cyclictest0-21swapper/321:05:193
3595870995316,19cyclictest3613970-21memory20:05:293
359587099531,24cyclictest3629806-21modprobe20:55:413
359587099531,24cyclictest3629806-21modprobe20:55:413
359587099531,20cyclictest3606235-21/usr/sbin/munin19:40:433
359587099531,20cyclictest3606235-21/usr/sbin/munin19:40:433
3595869995346,5cyclictest0-21swapper/222:01:012
3595869995346,5cyclictest0-21swapper/222:01:012
359586799531,24cyclictest3759029-21ssh00:25:100
359587099523,20cyclictest3610702-21sshd-auth19:55:283
3595870995217,32cyclictest0-21swapper/320:35:243
3595868995247,2cyclictest0-21swapper/121:57:571
3595867995245,5cyclictest86-21hwrng22:40:570
3595868995146,2cyclictest0-21swapper/121:39:311
3595868995145,2cyclictest0-21swapper/121:45:401
3595868995145,2cyclictest0-21swapper/121:45:401
3595870994820,3cyclictest0-21swapper/319:09:563
3595870994820,3cyclictest0-21swapper/319:09:563
359586799482,17cyclictest3747341-21/usr/sbin/munin00:06:000
3595868994641,2cyclictest0-21swapper/122:11:161
3595868994631,11cyclictest0-21swapper/122:41:081
3595868994530,11cyclictest0-21swapper/123:48:041
3595867994519,21cyclictest3669248-21ssh22:05:180
3595867994517,25cyclictest3702435-21ntpq22:55:530
359586799451,40cyclictest3762231-21ssh00:30:130
3595868994431,10cyclictest0-21swapper/100:10:191
3595867994437,4cyclictest86-21hwrng21:47:420
3595867994437,4cyclictest86-21hwrng21:47:420
359586799442,37cyclictest3749529-21ssh00:10:160
3595867994419,19cyclictest3624119-21sendmail-msp20:40:180
359586799441,38cyclictest3688824-21ssh22:35:240
3595869994336,4cyclictest0-21swapper/221:16:592
3595867994319,14cyclictest3766329-21ssh00:35:390
3595867994316,24cyclictest3634773-21ntpq21:10:470
359586799431,37cyclictest3721653-21ssh23:25:540
359586799431,31cyclictest3699176-21ssh22:50:590
359586799431,31cyclictest3699176-21ssh22:50:590
359586799428,31cyclictest614-21CodeMeterLin00:15:190
3595867994225,15cyclictest3729727-21ssh23:39:000
3595867994225,15cyclictest3729727-21ssh23:39:000
3595867994218,19cyclictest3739954-21ssh23:55:160
3595867994218,19cyclictest3646689-21/usr/sbin/munin21:30:070
3595867994217,21cyclictest3605301-21sendmail-msp19:40:200
3595867994217,21cyclictest3605301-21sendmail-msp19:40:200
3595867994216,16cyclictest3638772-21sshd-auth21:16:240
359586799421,37cyclictest3736701-21ssh23:50:110
359586799421,37cyclictest3679858-21ssh22:20:530
359586799421,36cyclictest3707863-21ssh23:05:180
3595867994213,25cyclictest3616524-21/usr/sbin/munin20:15:200
359586799421,23cyclictest3649982-21sh21:35:210
359586799421,19cyclictest3618064-21apt-get20:20:120
3595869994135,3cyclictest0-21swapper/221:53:512
3595868994128,10cyclictest0-21swapper/121:00:191
3595868994125,12cyclictest0-21swapper/122:59:571
3595868994123,13cyclictest0-21swapper/100:01:011
3595868994123,13cyclictest0-21swapper/100:01:011
3595867994117,20cyclictest3745342-21ssh00:02:580
3595867994117,20cyclictest3745342-21ssh00:02:580
359586799411,36cyclictest3653707-21ssh21:40:550
359586799411,36cyclictest3653707-21ssh21:40:550
359586799411,36cyclictest3607852-21ntpq19:45:400
359586799411,36cyclictest1759-21snmpd20:55:390
359586799411,36cyclictest1759-21snmpd20:55:390
359586799411,18cyclictest3715815-21/usr/sbin/munin23:16:020
3595869994032,4cyclictest0-21swapper/222:38:542
3595868994024,13cyclictest0-21swapper/100:20:121
3595868994024,12cyclictest0-21swapper/100:05:371
3595867994019,18cyclictest0-21swapper/023:40:210
3595867994019,17cyclictest3604708-21ntpq19:35:390
3595867994019,17cyclictest3598480-21ntp_states19:15:460
3595867994019,16cyclictest3659597-21ssh21:50:200
3595867994016,20cyclictest3685317-21ssh22:30:130
3595867994015,21cyclictest3640316-21apt-get21:20:090
3595867994014,22cyclictest3723920-21ssh23:30:170
359586799401,35cyclictest3695785-21ntpq22:45:360
359586799401,35cyclictest3695785-21ntpq22:45:360
359586799401,35cyclictest3666641-21ssh22:00:380
359586799401,35cyclictest3666641-21ssh22:00:380
359586799401,34cyclictest3611572-21sendmail-msp19:59:560
3595867994012,25cyclictest3596833-21/usr/sbin/munin19:10:360
359586799401,19cyclictest3644490-21ntpq21:25:480
3595869993932,4cyclictest0-21swapper/223:21:252
3595869993932,4cyclictest0-21swapper/223:21:252
3595868993923,12cyclictest0-21swapper/123:35:291
3595868993923,12cyclictest0-21swapper/123:35:291
3595868993923,12cyclictest0-21swapper/120:15:401
3595868993922,13cyclictest0-21swapper/123:25:321
3595867993922,13cyclictest3682594-21expr22:25:300
3595867993918,17cyclictest3611572-21sendmail-msp20:00:170
3595867993917,12cyclictest3614685-21kworker/u16:6+rpciod20:50:180
3595867993917,12cyclictest3614685-21kworker/u16:6+rpciod20:50:180
3595867993915,19cyclictest3623502-21ntpq20:35:370
3595867993914,16cyclictest3630379-21sendmail-msp21:00:110
359586799391,34cyclictest3601569-21ntpq19:25:400
3595867993911,25cyclictest3632880-21/usr/sbin/munin21:05:340
3595869993831,4cyclictest0-21swapper/223:00:252
3595869993831,4cyclictest0-21swapper/222:50:102
3595869993831,4cyclictest0-21swapper/222:50:102
3595869993831,4cyclictest0-21swapper/222:34:492
3595868993834,1cyclictest0-21swapper/122:08:111
3595868993821,13cyclictest0-21swapper/121:30:571
359586799381,33cyclictest3755591-21sendmail-msp00:20:310
359586799381,33cyclictest3733496-21ssh23:45:070
359586799381,33cyclictest3615028-21/usr/sbin/munin20:10:210
3595869993731,3cyclictest0-21swapper/223:06:332
3595869993731,3cyclictest0-21swapper/222:20:192
3595869993731,3cyclictest0-21swapper/221:44:382
3595869993731,3cyclictest0-21swapper/221:44:382
3595868993723,10cyclictest0-21swapper/122:35:551
3595868993722,11cyclictest0-21swapper/121:40:061
3595868993722,11cyclictest0-21swapper/121:40:061
3595867993718,15cyclictest3620352-21ntpq20:25:330
3595867993718,15cyclictest3603589-21sed19:30:570
359586799371,33cyclictest3712101-21ssh23:10:540
359586799371,33cyclictest3712101-21ssh23:10:540
359586799371,32cyclictest3706522-21kworker/u16:3+rpciod23:20:030
359586799371,32cyclictest3706522-21kworker/u16:3+rpciod23:20:030
359586799371,29cyclictest3625880-21/usr/sbin/munin20:45:070
3595869993630,3cyclictest0-21swapper/223:44:152
3595869993629,3cyclictest0-21swapper/223:30:202
3595869993624,7cyclictest0-21swapper/221:10:422
3595868993624,9cyclictest0-21swapper/120:00:321
3595868993623,10cyclictest0-21swapper/120:50:331
3595868993623,10cyclictest0-21swapper/120:50:331
3595868993622,10cyclictest0-21swapper/123:50:141
3595868993620,13cyclictest0-21swapper/121:20:211
359586799363,29cyclictest3609368-21ntpq19:50:340
3595867993627,6cyclictest3614537-21/usr/sbin/munin20:05:520
3595868993529,3cyclictest0-21swapper/123:00:131
3595868993529,3cyclictest0-21swapper/122:30:361
3595868993529,3cyclictest0-21swapper/121:11:591
3595868993521,11cyclictest0-21swapper/123:20:261
3595868993521,11cyclictest0-21swapper/123:20:261
3595868993519,11cyclictest0-21swapper/119:59:581
3595867993523,10cyclictest3622000-21ntpq20:30:430
3595867993523,10cyclictest3622000-21ntpq20:30:430
3595867993513,12cyclictest3599021-21sendmail-msp19:20:200
3595867993513,12cyclictest3599021-21sendmail-msp19:20:200
3595869993428,3cyclictest0-21swapper/223:10:412
3595869993428,3cyclictest0-21swapper/223:10:412
3595869993428,3cyclictest0-21swapper/200:00:322
3595869993428,3cyclictest0-21swapper/200:00:322
3595868993428,3cyclictest0-21swapper/123:40:311
3595868993428,3cyclictest0-21swapper/122:20:171
3595868993416,14cyclictest0-21swapper/122:49:591
3595868993416,14cyclictest0-21swapper/122:49:591
3595869993326,4cyclictest0-21swapper/222:55:562
3595869993326,4cyclictest0-21swapper/222:15:132
3595869993326,4cyclictest0-21swapper/222:15:132
3595868993328,3cyclictest0-21swapper/122:01:011
3595868993328,3cyclictest0-21swapper/122:01:011
3595868993327,3cyclictest0-21swapper/100:25:211
3595869993226,4cyclictest0-21swapper/220:00:182
3595869993226,4cyclictest0-21swapper/219:20:072
3595869993226,4cyclictest0-21swapper/219:20:072
3595869993225,4cyclictest0-21swapper/223:59:582
3595869993225,4cyclictest0-21swapper/222:45:522
3595869993225,4cyclictest0-21swapper/222:45:522
3595869993225,4cyclictest0-21swapper/200:05:062
3595869993222,6cyclictest0-21swapper/200:15:552
3595869993222,4cyclictest0-21swapper/200:29:562
3595869993220,8cyclictest0-21swapper/223:50:432
3595869993220,6cyclictest0-21swapper/222:40:372
3595868993226,3cyclictest0-21swapper/122:19:321
3595868993226,3cyclictest0-21swapper/122:19:321
3595868993226,3cyclictest0-21swapper/120:35:211
3595868993226,3cyclictest0-21swapper/120:20:171
3595868993225,4cyclictest0-21swapper/121:25:491
3595868993218,10cyclictest0-21swapper/119:44:581
3595868993218,10cyclictest0-21swapper/119:44:581
3595869993125,3cyclictest0-21swapper/219:45:412
3595869993123,5cyclictest0-21swapper/223:15:532
3595869993122,5cyclictest0-21swapper/221:25:292
3595869993120,6cyclictest0-21swapper/220:40:242
3595868993125,3cyclictest0-21swapper/121:16:001
3595868993125,3cyclictest0-21swapper/119:15:571
3595868993124,4cyclictest0-21swapper/100:19:571
3595869993024,3cyclictest0-21swapper/221:20:062
3595869993024,3cyclictest0-21swapper/200:30:502
3595869993023,4cyclictest0-21swapper/223:45:192
3595869993023,4cyclictest0-21swapper/219:25:222
3595869993023,4cyclictest0-21swapper/200:10:582
3595869993021,5cyclictest0-21swapper/200:20:382
3595869993021,4cyclictest0-21swapper/220:10:262
3595869993019,7cyclictest0-21swapper/223:25:082
3595869993018,7cyclictest0-21swapper/220:35:352
3595868993024,3cyclictest0-21swapper/123:10:321
3595868993024,3cyclictest0-21swapper/123:10:321
3595868993024,3cyclictest0-21swapper/123:05:151
3595868993024,3cyclictest0-21swapper/120:10:211
3595868993021,5cyclictest0-21swapper/123:31:081
3595869992922,4cyclictest0-21swapper/222:25:222
3595869992922,4cyclictest0-21swapper/219:59:562
3595869992922,4cyclictest0-21swapper/200:35:102
3595869992920,6cyclictest0-21swapper/221:30:072
3595868992923,3cyclictest0-21swapper/121:50:391
3595868992923,3cyclictest0-21swapper/120:40:101
3595868992923,3cyclictest0-21swapper/119:37:021
3595868992921,4cyclictest0-21swapper/100:33:031
3595869992823,3cyclictest0-21swapper/223:38:102
3595869992823,3cyclictest0-21swapper/223:38:102
3595869992823,3cyclictest0-21swapper/221:05:332
3595869992823,3cyclictest0-21swapper/220:05:382
3595869992823,3cyclictest0-21swapper/219:53:092
3595869992822,4cyclictest0-21swapper/219:40:122
3595869992822,4cyclictest0-21swapper/219:40:122
3595869992822,3cyclictest0-21swapper/219:30:222
3595869992821,4cyclictest0-21swapper/219:10:162
3595869992820,5cyclictest0-21swapper/220:21:242
3595869992819,5cyclictest0-21swapper/220:30:252
3595869992819,5cyclictest0-21swapper/220:30:252
3595868992822,3cyclictest0-21swapper/123:15:071
3595868992822,3cyclictest0-21swapper/122:51:141
3595868992822,3cyclictest0-21swapper/122:51:141
3595868992822,3cyclictest0-21swapper/122:29:371
3595868992822,3cyclictest0-21swapper/119:30:081
3595868992821,4cyclictest0-21swapper/123:57:281
3595868992821,4cyclictest0-21swapper/120:05:141
3595868992821,4cyclictest0-21swapper/119:50:251
3595869992722,3cyclictest0-21swapper/221:00:322
3595869992721,3cyclictest0-21swapper/220:45:352
3595869992720,4cyclictest0-21swapper/220:15:412
3595869992718,5cyclictest0-21swapper/220:25:332
3595868992721,3cyclictest0-21swapper/119:10:471
3595868992721,3cyclictest0-21swapper/100:35:141
3595868992720,4cyclictest0-21swapper/119:20:491
3595868992720,4cyclictest0-21swapper/119:20:491
3595869992621,3cyclictest0-21swapper/220:50:462
3595869992621,3cyclictest0-21swapper/220:50:462
3595869992619,4cyclictest0-21swapper/219:35:342
3595868992620,3cyclictest0-21swapper/121:05:091
3595868992620,3cyclictest0-21swapper/120:45:271
3595868992620,3cyclictest0-21swapper/120:30:421
3595868992620,3cyclictest0-21swapper/120:30:421
3595868992620,3cyclictest0-21swapper/120:26:251
3595868992620,3cyclictest0-21swapper/119:25:441
3595867992618,6cyclictest3591734-21kworker/u16:4+rpciod19:09:560
3595867992618,6cyclictest3591734-21kworker/u16:4+rpciod19:09:560
3595869992519,3cyclictest0-21swapper/219:15:172
3595868992519,3cyclictest0-21swapper/119:45:271
3595869992316,4cyclictest0-21swapper/219:09:572
3595868992216,3cyclictest0-21swapper/119:09:571
3595868992216,3cyclictest0-21swapper/119:09:571
3595869992111,5cyclictest0-21swapper/219:09:562
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional