-
Notifications
You must be signed in to change notification settings - Fork 346
Add support for generating inventory with a graph query #1994
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
I'm not sure why I'm getting this error in the sanity test though.. And only for 3.11 |
|
Our test environment did not rely on pandas, so this error did not occur. If an error occurs in python3.11, does it mean that pandas is not included in the python3.11 standard library and needs to be installed by oneself? |
This comment was marked as resolved.
This comment was marked as resolved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what caused it. Although the sanity detection indicates that 'pandas' does not exist, but there is no problem when azuer_kql.py is executed. So I want to temporarily solve the current problem in this way. Thank you!
New Azure Inventory Plugin Using Resource Graph and KQL for Improved Performance in Environments with hundreds of subscriptions.
|
Hi, Greetings |
|
For the MSFT people, the support case number was 2406170050002212 I'll quote from it for the red hat people: """ I think the answer to your question “Does Microsoft want to change that?” is “No, we don’t want to change how this works right now”. Andreea discussed the case with both ARG and Compute resource groups. They found the reason for the missing property but due to the design, there is nothing that can be changed without re-architecting some major Azure components and this is not something we’re ready for at this point. Maybe in the future, but for now we’ll have to live with validating the properties on the client side. In the case I was exploring I noticed that important fields for an inventory (in this case current powerstate) would silently fail and just show "" instead of the actual powerstate. I have since reverted to using the API directly. This, at least from the information in that case, is the only interface on which microsoft provides "guarantees" for information accuracy. |
|
It also goes the other way, just on Friday I had another case with long-deleted resources still showing up in Graph. I stopped monitoring the situation after 8h post-delete because my day was finished. The answer from support is simply "this is a known issue" -- so also in that area, don't rely on Graph. If you want a proper inventory you have to use API until MSFT rethinks what Graph is and what it should guarantee to it's users |
I will put something in the notes. I also have an inquiry to the customer who is currently generating their inventory via ARG and if they are aware of this issue. |
|
the official Resource Graph answer is "for new resource it can take up to 30 hours to appear in graph" and same for deleted resources "it can take up to 30 hours until a deleted resource disappears from graph" |
|
I've put a disclaimer in the inventory module. |
| for how to craft your own query. The one requirement is that you need to provide inventory_hostname. | ||
| - Requires a YAML configuration file whose name ends with 'azure_kql.(yml|yaml)' | ||
| - Be aware that currently Azure Resource Graph may not be consistent with the actual state of your | ||
| resources. It can take up to 30 minutes for updates to propagate. This applies both for resources |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you misread my comment :) 30 hours not minutes.
|
I've made a fix and some additions to warning in #2002 |
SUMMARY
New Azure Inventory Plugin Using Resource Graph and KQL for Improved Performance in Environments with hundreds of subscriptions.
ISSUE TYPE
COMPONENT NAME
plugins/doc_fragments/azure_kql.py
plugins/inventory/azure_kql.py
requirements.txt
tests/integration/targets/inventory_azure/playbooks/create_inventory_config.yml
tests/integration/targets/inventory_azure/playbooks/empty_inventory_config.yml
tests/integration/targets/inventory_azure/playbooks/setup.yml
tests/integration/targets/inventory_azure/playbooks/test_inventory_kql.yml
tests/integration/targets/inventory_azure/runme.sh
tests/integration/targets/inventory_azure/templates/kql.yml
ADDITIONAL INFORMATION
The original idea came from a customer which had written a custom python script to generate a static inventory file. We expanded on that and made it into a proper inventory plugin.
While there is some overlap with the current azure_rm inventory plugin I think this is different enough to warrent a separate plugin. The query format gives a lot more control to the user to be able to generate inventory files.
Integration tests have been updated to handle this new module as well.