You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-11 - 04:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sat Jul 11, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18445996748,11cyclictest35966irq/58-eth021:20:113
1843199624,37cyclictest10166-21ssh23:35:102
1843199601,35cyclictest0-21swapper/220:25:132
1843199542,28cyclictest28298-21ssh23:45:122
1844599535,27cyclictest35966irq/58-eth022:55:133
18445995032,9cyclictest35966irq/58-eth021:55:103
1842299503,37cyclictest27325-21ssh00:20:131
1844599481,44cyclictest0-21swapper/322:10:123
18408994813,8cyclictest131rcu_preempt19:10:110
1843199461,33cyclictest17027-21ssh22:10:122
1842299462,33cyclictest8868-21ssh21:30:151
1842299461,33cyclictest16229-21ssh22:45:111
1844599455,32cyclictest35966irq/58-eth023:10:123
1842299453,8cyclictest131rcu_preempt20:00:131
18408994511,20cyclictest12-21ksoftirqd/020:20:110
18408994317,9cyclictest131rcu_preempt20:35:120
18408994317,9cyclictest131rcu_preempt20:35:120
1840899430,32cyclictest22205-21ssh00:35:120
1840899424,33cyclictest26996-21sh21:40:120
18408994125,12cyclictest0-21swapper/023:30:130
1840899411,6cyclictest131rcu_preempt19:25:130
1844599402,22cyclictest35966irq/58-eth023:55:123
18445993922,12cyclictest35966irq/58-eth021:15:133
1842299391,30cyclictest32094-21ssh21:25:131
1840899395,8cyclictest131rcu_preempt19:50:120
1842299381,28cyclictest13711-21ssh21:15:151
1840899384,9cyclictest131rcu_preempt19:15:110
1842299371,23cyclictest1083-21ssh23:30:121
1840899373,30cyclictest29428-21ssh23:10:120
18408993731,3cyclictest0-21swapper/022:50:130
1840899372,6cyclictest131rcu_preempt20:25:120
1840899372,29cyclictest21238-21ssh22:30:130
1840899361,27cyclictest23303-21ssh00:00:120
18408993610,10cyclictest131rcu_preempt19:20:130
1844599351,20cyclictest35966irq/58-eth000:25:133
1840899357,21cyclictest12-21ksoftirqd/000:30:130
1842299340,27cyclictest0-21swapper/122:30:111
1840899349,22cyclictest3681-21ssh21:45:110
1844599331,5cyclictest0-21swapper/320:05:133
1843199333,8cyclictest4058-21diskmemload22:55:132
18431993326,4cyclictest27-21ksoftirqd/222:20:132
1843199332,23cyclictest27346irq/40-dwc2_hso19:50:132
1840899333,19cyclictest151rcuc/022:40:130
1840899333,19cyclictest151rcuc/022:40:120
18408993312,13cyclictest12-21ksoftirqd/022:55:120
1844599322,4cyclictest0-21swapper/300:20:143
18445993219,9cyclictest35966irq/58-eth022:35:133
1844599321,2cyclictest0-21swapper/323:05:123
1844599321,19cyclictest35966irq/58-eth000:10:113
1843199321,21cyclictest21994-21ssh21:55:132
1842299322,15cyclictest18170-21ssh00:15:131
1840899322,24cyclictest24365-21ssh23:25:120
18445993118,9cyclictest35966irq/58-eth022:20:133
18445993116,10cyclictest0-21swapper/319:55:143
1844599311,4cyclictest35966irq/58-eth000:35:123
1844599302,5cyclictest0-21swapper/319:50:113
1844599301,4cyclictest0-21swapper/322:30:133
1843199302,21cyclictest0-21swapper/220:10:182
1840899304,21cyclictest17940-21ssh21:35:140
18408993010,18cyclictest12-21ksoftirqd/023:40:130
1844599293,19cyclictest35966irq/58-eth021:35:143
1844599292,2cyclictest0-21swapper/323:00:133
1844599292,15cyclictest35966irq/58-eth022:50:133
1844599291,5cyclictest0-21swapper/320:35:123
1844599291,5cyclictest0-21swapper/320:35:123
1844599291,17cyclictest35966irq/58-eth021:30:143
1843199291,15cyclictest12104-21ssh22:25:112
1840899291,20cyclictest844-21munin-node19:30:140
1844599282,4cyclictest0-21swapper/320:25:123
1844599282,4cyclictest0-21swapper/319:20:133
1844599282,3cyclictest0-21swapper/321:05:133
1843199282,20cyclictest0-21swapper/220:20:132
1843199281,18cyclictest27335-21ssh00:20:132
1842299283,16cyclictest22890-21ssh21:20:121
1840899284,17cyclictest8417-1kworker/u9:000:10:110
1844599277,12cyclictest35966irq/58-eth000:15:143
1844599271,4cyclictest0-21swapper/300:00:123
1843199271,20cyclictest17944-21ssh21:35:142
1842299273,9cyclictest131rcu_preempt23:00:131
1842299271,19cyclictest19320-21ssh23:40:141
1842299271,17cyclictest30329-21ssh22:35:121
1840899274,18cyclictest4058-21diskmemload00:20:140
18408992718,6cyclictest17032-21ssh22:10:130
18408992717,8cyclictest151rcuc/023:00:120
18408992716,8cyclictest28314-21sh23:45:130
1844599263,2cyclictest0-21swapper/323:37:223
1844599261,3cyclictest0-21swapper/323:54:463
1844599261,3cyclictest0-21swapper/323:32:243
1844599261,3cyclictest0-21swapper/323:29:193
1844599261,3cyclictest0-21swapper/323:23:003
1844599261,3cyclictest0-21swapper/322:40:343
1844599261,3cyclictest0-21swapper/322:40:343
1844599261,3cyclictest0-21swapper/321:28:173
1844599261,3cyclictest0-21swapper/300:34:263
1844599261,3cyclictest0-21swapper/300:08:143
1844599261,2cyclictest0-21swapper/322:20:023
1844599261,2cyclictest0-21swapper/321:12:023
1844599261,2cyclictest0-21swapper/320:45:133
1844599261,22cyclictest0-21swapper/321:45:103
1844599261,16cyclictest35966irq/58-eth019:10:123
1843199269,14cyclictest2544-21cat22:00:282
1843199264,3cyclictest0-21swapper/221:00:132
1843199262,9cyclictest0-21swapper/221:25:142
1843199262,18cyclictest27346irq/40-dwc2_hso20:05:122
1843199262,13cyclictest27346irq/40-dwc2_hso19:10:112
1842299269,3cyclictest131rcu_preempt21:45:251
1842299261,20cyclictest628-21tcf-agent23:05:261
18408992618,3cyclictest131rcu_preempt22:35:110
18408992617,5cyclictest4979-21sh23:50:120
1844599252,4cyclictest0-21swapper/320:20:133
1844599251,4cyclictest0-21swapper/322:25:263
1844599251,4cyclictest0-21swapper/319:45:153
1844599251,3cyclictest0-21swapper/323:45:193
1844599251,3cyclictest0-21swapper/323:40:023
1844599251,3cyclictest0-21swapper/323:15:123
1844599251,3cyclictest0-21swapper/323:15:123
1844599251,3cyclictest0-21swapper/322:45:183
1844599251,3cyclictest0-21swapper/321:50:093
1844599251,3cyclictest0-21swapper/321:50:093
1844599251,3cyclictest0-21swapper/320:30:293
1843199252,15cyclictest6156-21ssh23:15:132
1843199252,15cyclictest6156-21ssh23:15:122
1843199251,12cyclictest32410-21ssh00:05:132
1840899253,15cyclictest0-21swapper/000:15:140
1840899251,17cyclictest12-21ksoftirqd/020:45:130
1840899250,17cyclictest20222-21ssh23:05:050
1844599242,16cyclictest35966irq/58-eth022:00:123
1844599241,3cyclictest0-21swapper/321:41:173
1844599241,3cyclictest0-21swapper/319:25:233
1844599241,3cyclictest0-21swapper/319:15:153
1844599241,2cyclictest0-21swapper/322:05:273
1843199243,17cyclictest22906-21ssh21:20:132
1843199243,16cyclictest27346irq/40-dwc2_hso23:40:132
1843199242,17cyclictest26218-21ssh22:15:142
1843199242,14cyclictest27346irq/40-dwc2_hso00:00:122
1843199241,14cyclictest4467-21sh21:10:132
1842299241,18cyclictest32407-21sh00:05:141
18408992417,4cyclictest131rcu_preempt21:15:130
1840899241,18cyclictest2989-21rm22:20:130
1840899241,17cyclictest12841-21ssh21:50:110
1840899241,17cyclictest12841-21ssh21:50:110
1840899241,15cyclictest26217-21sh22:15:140
1844599231,5cyclictest0-21swapper/320:15:253
1844599231,3cyclictest0-21swapper/321:00:153
1844599231,3cyclictest0-21swapper/320:55:283
1844599231,3cyclictest0-21swapper/320:40:233
1844599231,3cyclictest0-21swapper/320:10:193
1844599231,3cyclictest0-21swapper/320:00:183
1844599231,3cyclictest0-21swapper/319:40:133
1844599231,3cyclictest0-21swapper/319:30:273
1843199234,13cyclictest27346irq/40-dwc2_hso21:05:122
1843199233,15cyclictest27346irq/40-dwc2_hso23:25:112
1843199233,15cyclictest27346irq/40-dwc2_hso20:50:112
1843199232,16cyclictest0-21swapper/200:38:592
18431992312,9cyclictest18411-21ssh00:30:582
18431992312,5cyclictest26252-21ssh23:06:432
1843199230,20cyclictest24711-21ls22:30:232
1843199230,19cyclictest3971-21idleruntime21:45:152
1842299238,7cyclictest131rcu_preempt21:40:011
18422992313,7cyclictest29174-21cron19:55:011
1840899231,4cyclictest131rcu_preempt22:45:130
1844599221,3cyclictest0-21swapper/320:50:273
1844599221,3cyclictest0-21swapper/319:35:163
1843199229,10cyclictest7152-21kthreadcore19:30:212
1843199224,9cyclictest15142-21timerandwakeup23:00:292
1843199223,15cyclictest27346irq/40-dwc2_hso20:55:122
1843199222,15cyclictest0-21swapper/223:20:142
1843199222,12cyclictest27346irq/40-dwc2_hso22:40:142
1843199222,12cyclictest27346irq/40-dwc2_hso22:40:142
1843199222,11cyclictest31291-21grep19:55:212
1843199222,10cyclictest26569-21ssh22:50:202
1843199221,9cyclictest5849-21latency_hist23:15:002
1843199221,8cyclictest3963-21ssh00:25:142
18431992211,9cyclictest21385-21ssh23:58:152
18431992211,9cyclictest20895-21cron20:20:012
18431992211,5cyclictest7819-21ssh23:32:402
1843199221,15cyclictest28170-21find19:20:122
1842299225,12cyclictest4980-21ssh23:50:111
1842299221,17cyclictest13075-21ssh00:30:141
1842299221,15cyclictest21995-21sh21:55:131
1840899222,13cyclictest15234-21ssh23:20:140
1840899221,6cyclictest2972966ssh21:58:520
18408992211,4cyclictest12-21ksoftirqd/021:33:430
1840899220,3cyclictest131rcu_preempt23:35:150
1843199214,10cyclictest11103-21kthreadcore22:05:232
1843199213,10cyclictest27346irq/40-dwc2_hso20:30:122
1843199212,9cyclictest11365-21kthreadcore21:30:232
1843199212,10cyclictest12959-21diskstats21:50:132
1843199212,10cyclictest12959-21diskstats21:50:132
18431992112,6cyclictest19559-21cron19:45:012
1843199211,16cyclictest4102-21grep19:25:282
1843199211,16cyclictest14927-21latency_hist19:40:012
1843199211,11cyclictest21257-21kthreadcore19:45:212
1843199211,10cyclictest30062-21kthreadcore21:40:212
1842299214,11cyclictest31136-21ssh22:00:121
18422992113,5cyclictest10140-21/usr/sbin/munin23:35:111
18422992112,6cyclictest690-21cron23:30:021
1842299211,15cyclictest7893-21ssh22:05:141
1842299210,18cyclictest23367-21ls00:00:131
1840899211,5cyclictest53646usb-storage20:15:050
1840899211,16cyclictest3414-21latency00:05:230
1840899211,13cyclictest24400-21latency_hist19:50:020
1843199202,9cyclictest1207-21kthreadcore22:35:222
1843199202,15cyclictest14365-21kthreadcore20:45:202
1843199202,10cyclictest17120-21kthreadcore22:45:192
1843199201,9cyclictest5623-21sed23:50:182
1843199201,9cyclictest2025-21cut20:00:162
1843199201,9cyclictest20204-21kthreadcore00:15:222
18431992013,5cyclictest17493-21ssh00:14:422
1843199201,11cyclictest3286-21grep20:35:152
1843199201,11cyclictest3286-21grep20:35:152
1843199201,11cyclictest23359-21apt-get19:15:132
1843199201,10cyclictest13713-21cpu21:15:152
1842299208,3cyclictest131rcu_preempt21:40:201
1842299207,10cyclictest10779-21sendmail23:00:011
1842299205,7cyclictest0-21swapper/100:10:121
1842299202,15cyclictest3105-21apt-get20:35:121
1842299202,15cyclictest3105-21apt-get20:35:111
18422992014,4cyclictest0-21swapper/121:05:241
1842299201,12cyclictest29920-21ssh22:15:271
1842299201,10cyclictest0-21swapper/120:10:251
18422992010,3cyclictest21-21ksoftirqd/123:55:131
18408992011,7cyclictest14749-21cron19:40:020
1840899201,16cyclictest4454-21apt-get21:10:110
1840899201,15cyclictest14123-21rm23:55:130
1840899200,17cyclictest29394-21find19:55:120
1844599191,3cyclictest0-21swapper/319:10:013
1843199191,9cyclictest8606-21ps20:40:192
1843199191,9cyclictest18504-21latency_hist19:10:012
18422991914,3cyclictest0-21swapper/120:25:211
1842299191,14cyclictest4468-21ssh21:10:131
1842299191,14cyclictest25406-21rm22:50:141
1842299191,14cyclictest0-21swapper/120:45:141
1842299191,12cyclictest26059-21timerwakeupswit20:55:271
1840899193,12cyclictest32462-21kthreadcore22:00:200
18408991913,4cyclictest0-21swapper/023:15:250
18408991913,4cyclictest0-21swapper/023:15:250
1840899191,15cyclictest7096-21ssh00:25:220
1840899191,14cyclictest30746-21apt-get20:30:120
1842299185,10cyclictest0-21swapper/120:30:121
1842299184,11cyclictest14243-21kthreadcore21:50:181
1842299184,11cyclictest14243-21kthreadcore21:50:181
1842299182,13cyclictest10479-21ssh23:15:311
1842299182,13cyclictest10479-21ssh23:15:311
1842299182,12cyclictest31988-21ntp_states23:45:261
1842299182,11cyclictest15005-21apt-get19:40:101
1842299182,10cyclictest26111-21ls00:35:291
1842299181,15cyclictest8772-21grep19:30:251
1842299181,10cyclictest9544-21kthreadcore20:40:191
18422991810,4cyclictest0-21swapper/122:40:141
18422991810,4cyclictest0-21swapper/122:40:131
1840899189,5cyclictest24903-21kthreadcore21:20:200
1840899188,8cyclictest471-21tr21:25:200
1840899187,2cyclictest12-21ksoftirqd/020:15:010
1840899184,11cyclictest25159-21kthreadcore20:55:210
1840899182,13cyclictest11239-21grep20:10:010
1840899181,2cyclictest131rcu_preempt22:25:290
18408991812,4cyclictest0-21swapper/021:05:180
1840899181,12cyclictest8694-21cut22:05:190
1840899181,11cyclictest12-21ksoftirqd/021:00:130
18408991810,6cyclictest12465-21sshd20:40:450
1842299175,9cyclictest17372-21if_eth122:10:151
1842299174,10cyclictest29400-21/usr/sbin/munin23:10:281
1842299171,7cyclictest18208-21kthreadcore20:15:191
1842299171,13cyclictest31928-21kthreadcore19:55:211
1842299171,13cyclictest2960-21apt-get22:20:111
1842299171,12cyclictest3957-21sh00:25:121
1842299171,12cyclictest15926-21sed23:20:181
1840899178,7cyclictest20817-21memory20:50:220
1842299169,4cyclictest17857-21irqcore20:50:161
1842299168,6cyclictest29131-21cut21:00:211
1842299168,5cyclictest20597-21kthreadcore19:45:201
1842299163,10cyclictest12070-21/usr/sbin/munin22:25:141
1842299162,5cyclictest28185-21apt-get19:20:121
1842299162,11cyclictest27959-21idleruntime-cro19:20:011
1842299162,11cyclictest21583-21irqcore20:20:181
1842299162,11cyclictest18458-21latency_hist19:10:011
1842299162,11cyclictest10462-21irqcore19:35:181
1842299162,10cyclictest0-21swapper/120:05:141
1842299161,12cyclictest3696-21tr19:25:221
1842299161,12cyclictest18550-21apt-get19:10:111
1840899161,3cyclictest131rcu_preempt19:40:120
1840899160,2cyclictest131rcu_preempt20:00:260
1840899141,10cyclictest18412-21latency_hist19:10:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional